X-ray imaging apparatus and image processing method

Abstract

To detect the body movement of a subject during photographing a fluoroscopic image.SOLUTION: An image of a subject is captured by a camera provided in the X-ray imaging apparatus. First, the skeleton of a human figure included in an image is detected, and the skeleton of a subject placed on the top plate of an X-ray imaging apparatus is extracted based on the detected skeleton. A movement amount (optical flow) of corresponding pixels is calculated between the captured image and an image acquired by the camera at a timing earlier than a timing at which the captured image is acquired by a predetermined time. A plurality of regions of interest are set in the image of the subject based on the position of the skeleton of the subject. The number of regions of interest in which the movement amount of the pixel of the subject in the region of interest is larger than a predetermined threshold is obtained, and when the obtained number of regions of interest is larger than a predetermined value, it is determined that the body movement occurs in the subject.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an X-ray imaging apparatus that irradiates an object with X-rays to obtain an image. [Background technology]

[0002] IVR (Interventional Radiology) is a well-known technique in which an endoscope or catheter is inserted into a patient to perform various procedures while irradiating the patient with X-rays from an X-ray imaging device to obtain fluoroscopic images. Doctors perform procedures on the patient while viewing monitors displaying fluoroscopic images and endoscopic images.

[0003] When checking the patient's condition while performing a procedure while looking at fluoroscopic images, doctors either move their gaze to the patient's face, feet, or hands to check visually, or move their gaze to a monitor displaying images from a camera attached to the ceiling or upper part of the wall in the room to check the patient through the camera image.

[0004] On the other hand, Patent Document 1 discloses a technique for detecting the body movement of a subject by processing a fluoroscopic image.

[0005] Patent Document 2 discloses a technology that uses a camera to photograph multiple workers performing tasks, processes the images, and performs skeletal detection to identify a worker performing a specific task from among the multiple workers. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2021-186199 [Patent Document 2] Patent Publication No. 2021-196783 Summary of the Invention [Problem to be solved by the invention]

[0007] IVR involves inserting endoscopes and catheters into patients while acquiring fluoroscopic images and performing various procedures on them. Patients are sedated during this procedure, but the patient may unconsciously try to remove the endoscope or move. If this occurs, the physician must call a nurse or other staff member inside or outside the examination room while performing the procedure, and issue instructions such as holding the patient down. Therefore, even while performing the procedure and watching the endoscope monitor, the physician must be aware of the patient's movements.

[0008] However, in order for a doctor to visually check the condition of the patient during surgery, he or she must move his or her line of sight from the endoscope monitor to the patient, which places a burden on the doctor.

[0009] On the other hand, when a doctor checks the patient's condition by looking at a monitor showing camera images from inside the room, it is difficult to instantly grasp the patient's condition because the camera image also shows doctors, technicians, and nurses in addition to the patient.

[0010] An object of the present invention is to detect the body movement of a subject during fluoroscopic imaging. [Means for solving the problem]

[0011] To achieve the above object, the present invention provides an X-ray imaging device including a tabletop on which a subject is placed, an X-ray irradiator that irradiates X-rays onto the subject, an X-ray detector that detects X-rays irradiated from the X-ray irradiator and transmitted through the subject, and a body motion detection unit that receives images from a camera that captures the subject and processes the images to detect the occurrence of body motion of the subject. The body motion detection unit includes a skeleton detection unit, an optical flow calculation unit, a region of interest setting unit, and a determination unit. The skeleton detection unit detects the skeleton of a human figure included in the image and extracts the skeleton of the subject placed on the tabletop based on the detected skeleton. The optical flow calculation unit calculates the amount of movement of corresponding pixels between an image and an image captured by the camera at a predetermined time before the image was captured. The region of interest setting unit sets multiple regions of interest on the image of the subject based on the position of the subject's skeleton extracted by the skeleton detection unit. The determination unit determines the number of regions of interest in which the amount of movement of the subject's pixels within the region of interest is greater than a predetermined threshold. If the number of regions of interest thus determined is greater than a predetermined value, it is determined that a body movement has occurred in the subject. [Effects of the Invention]

[0012] According to the present invention, it is possible to detect the body movement of a subject while capturing fluoroscopic images, and by notifying the physician, the physician can easily grasp the occurrence of the patient's body movement even during a procedure. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 2A is a front view of the X-ray imaging device 1 according to the first embodiment of the present invention, and FIGS. 2B and 2C are diagrams showing examples of images displayed on a camera image display monitor. [Figure 2] FIG. 2 is a functional block diagram of the X-ray imaging apparatus according to the first embodiment. [Figure 3] 1 is a diagram illustrating an example of the arrangement of a tabletop 80 of an X-ray imaging device 1, a subject 10, and doctors and nurses in a typical examination room. [Figure 4] 3A and 3B are diagrams showing an image captured by a camera of the X-ray imaging device 1 of the first embodiment and a skeleton detected from the image. [Figure 5]10 is a diagram for explaining that a rectangular area 51 that substantially covers the subject 10 is set in an image captured by a camera of the X-ray imaging device of the first embodiment. FIG. [Figure 6] 6 is a diagram for explaining how a rectangular region 51 in FIG. 5 is divided into a grid to set a region of interest 52. FIG. [Figure 7] 3 is a diagram showing an image in which pixel values ​​are optical flows (amounts of movement) calculated from images captured by the camera of the X-ray imaging device of the first embodiment. FIG. [Figure 8] 7 is a diagram for explaining a state in which the region of interest 52 in FIG. 6 is set on an image in which pixel values ​​are optical flows (amounts of movement) from an image taken by the X-ray imaging device of the first embodiment. [Figure 9] 4 is a flowchart showing the operation of the X-ray imaging apparatus of the first embodiment. [Figure 10] 10 is a diagram showing a state in which a notification dialog 161 notifying the occurrence of body movement and a graph 162 are displayed on the display screen of the camera image display monitor 15 of the X-ray imaging apparatus of the first embodiment. FIG. [Figure 11] FIG. 12 is an enlarged view of graph 162 in FIG. 11. [Figure 12] 10(a) and 10(b) are diagrams showing that when an image 21a is displayed on the display screen of the camera image display monitor 15 of the X-ray imaging device of the first embodiment, the occurrence of a body movement is notified by the color of the edge of the image 21a. [Figure 13] 10 is a diagram illustrating an example in which an image of a doctor's hand and an image of an endoscope are included in a region of interest 52 of the first embodiment when the region of interest 52 is set on a subject 10. FIG. [Figure 14] FIG. 10 is a diagram for explaining regions of interest 71 and 72 set in the second embodiment. [Figure 15] 10 is a flowchart showing the operation of the X-ray imaging apparatus of the second embodiment. [Figure 16] (a) and (b) are diagrams showing that when an image 21a is displayed on the display screen of the camera image display monitor 15 of the X-ray imaging device of embodiment 2, frames indicating regions of interest 71, 72 are displayed and the occurrence of body movement is notified by the color of the regions of interest. [Figure 17] FIG. 10 is a diagram for explaining regions of interest 71 and 78 set in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] An X-ray irradiation apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

[0015] <<Embodiment 1>> An X-ray imaging device 1 of embodiment 1 will be described with reference to the drawings. FIG. 1(a) is a front view of the X-ray imaging device 1, and FIGS. 1(b) and 1(c) are images displayed on a camera image display monitor. FIG. 2 is a functional block diagram of the X-ray imaging device 1. FIG. 3 is a diagram showing the arrangement of the X-ray imaging device, the subject, and doctors and nurses in an examination room. FIGS. 4 to 8 are examples of images captured by a camera and image processing. FIG. 9 is a flowchart showing the operation of the X-ray imaging device 1, and FIGS. 10 to 12 are diagrams showing examples of displaying body movement detection results.

[0016] As shown in FIG. 1(a), the X-ray imaging device 1 includes a tabletop 80 on which a subject 10 is placed, and an X-ray irradiator 11 that irradiates the subject 10 with X-rays. The X-ray irradiator 11 is supported on a stand 70 by an X-ray support unit 50. The stand 70 is further equipped with an X-ray up / down movement mechanism 60 that moves the X-ray support unit 50 up and down, a longitudinal direction movement mechanism (not shown) that moves the X-ray support unit 50 in the longitudinal direction 50a of the tabletop 80, a lateral direction movement mechanism (not shown) that moves the X-ray support unit 50 in the lateral direction 50b of the tabletop 80, and an X-ray tilt mechanism 61 that tilts (50c) the central axis of the X-ray support unit 50 around the lateral direction of the tabletop. The X-ray support unit 50 is further equipped with an X-ray rotation mechanism (not shown) that rotates the X-ray irradiator 11 around the longitudinal direction of the tabletop. The X-ray support part 50 is provided with a pressing tool 62 that is used to press the abdomen of the subject 10 during imaging.

[0017] On the other hand, the top plate 80 is supported by a top plate support portion 81 relative to the stand 70 .

[0018] An X-ray detector 12 is disposed inside the top board support part 81. The X-ray detector 12 is a planar detector in which X-ray detection elements are arranged two-dimensionally. The X-ray detector 12 detects X-rays that are irradiated from the X-ray irradiator 11 and have passed through the subject 10.

[0019] The X-ray support part 50 or the X-ray irradiator 11 is provided with a camera 21 that captures an image of the subject 10. The camera 21 is provided at a position that does not block the X-rays emitted from the X-ray irradiator 11. The camera 21 has an angle of view set so that the entire body of the subject 10 on the tabletop 80 is captured, as shown in FIGS. 1(b) and 1(c). The camera 21 may be fixed to the cover (housing) of the X-ray irradiator 11.

[0020] When the X-ray support part 50 moves in the longitudinal direction 50a of the tabletop 80, the angle of view of the camera 21 moves accordingly as shown in Figure 1(b), and when the X-ray support part 50 moves in the lateral direction 50b of the tabletop 80, the angle of view of the camera 21 moves accordingly as shown in Figure 1(c).

[0021] As shown in Fig. 2, an X-ray image generating unit 13 is connected to the X-ray detector 12. The X-ray image generating unit 13 receives signals output by each X-ray detection element of the X-ray detector 12 upon detecting X-rays, and generates X-ray images (fluoroscopic images in this case) at a predetermined frame rate. A fluoroscopic image display monitor 14 is connected to the X-ray image generating unit 13. The fluoroscopic images are displayed on the fluoroscopic image display monitor 14.

[0022] Meanwhile, the body movement detection unit 20 is connected to the camera 21. The body movement detection unit 20 receives images from the camera 21 and processes the images. Specifically, the body movement detection unit 20 receives images from the camera 21 and detects the skeletons of all human images included in the received images. The body movement detection unit 20 calculates predetermined feature amounts based on the detected skeletons, and extracts an image of the subject 10 sitting on the tabletop 80 from among the multiple human images based on the calculated feature amounts. The body movement detection unit 20 displays the extracted image of the subject 10 on the connected camera image display monitor 15.

[0023] More specifically, the body movement detection unit 20 includes an image capture unit 22, an image rotation unit 23, a skeleton detection unit 24, a subject skeleton extraction unit 25, a region of interest setting unit 26, an optical flow calculation unit 27, and a determination unit 28.

[0024] The image capturing unit 22 captures the images 21a output by the camera 21 at predetermined time intervals. The predetermined time intervals can be set to be approximately the same as the frame rate of the fluoroscopic images generated by the X-ray image generating unit 13. The images 21a output by the camera 21 are images captured such that the body axis of the subject on the tabletop 80 is the horizontal axis of the image, as shown in FIGS. 1(b) and 1(c).

[0025] The image rotation unit 23 rotates the image 21a by 90 degrees in a predetermined direction so that the head position of the image of the subject 10 on the tabletop 80 is at the top of the image 21a, as shown in Fig. 4. Since it is predetermined which end of the tabletop 80 in the longitudinal direction the head of the subject 10 is placed on, the direction in which the image 21a is rotated is determined in advance by the predetermined head position on the tabletop 80 and the orientation of the camera 21.

[0026] 3, an endoscope device is placed near a tabletop 80 in the examination room, and in order to insert an endoscope into the subject 10 and perform a procedure, medical personnel such as doctors and nurses stand or sit in chairs around the tabletop 80. Therefore, as shown in FIGS. 1(b) and 1(c), images 21a captured by camera 21 include not only the subject 10 but also the medical personnel.

[0027] The skeleton detection unit 24 extracts skeletal feature points (ears, noses, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc.) of all human figures in the image 21a as shown in FIG. 3. The skeleton detection unit 24 detects the skeletons of all human figures by detecting their two-dimensional coordinates. The skeleton detection unit 24 can be realized by a known pose detection algorithm. For example, the pose estimation algorithm Open Pose (registered trademark) included in Open VINO (registered trademark) provided by Intel Corp. can be used.

[0028] The subject skeleton extraction unit 25 calculates predetermined features using the skeletal feature points of all human images detected by the skeleton detection unit 24, and extracts the skeleton of the subject 10 placed on the tabletop based on the calculated features.

[0029] For example, subject skeleton extraction unit 25 determines as subject candidates those whose shoulders and ears are detected from the skeletons of all detected human images. For each subject candidate, subject skeleton extraction unit 25 determines midpoint 41 of the line connecting both shoulders of the subject candidate, as shown in FIG. 4, and calculates the distance between the determined midpoint 41 of both shoulders and center 45 of image 21a. The subject candidate with the shortest distance between midpoint 41 of both shoulders and center 45 of image 21a is determined to be subject 10.

[0030] Furthermore, if there are two or more subject candidates who have the same distance between the midpoint 41 of both shoulders and the center 45 of the image 21a as the above-mentioned feature, the subject skeleton extraction unit 25 further calculates the angle θ that the line connecting both shoulders of the subject candidate makes with the horizontal direction of the image 21a (parallel to the short side direction of the tabletop 80) as a feature, and determines the subject candidate whose angle θ of the line connecting both shoulders is close to zero, i.e., close to the horizontal direction of the image 21a, to be the subject 10.

[0031] In addition, the subject skeleton extraction unit 25 may determine that the subject 10 is not present if the distance between the midpoint 41 of both shoulders of the skeleton determined to be the subject 10 and the center 45 of the image 21a differs by more than a predetermined value from the average of the distances between the midpoint 41 of both shoulders of the subject 10 and the center 45 of the image 21a in multiple images 21a taken by the camera 21 since the start of fluoroscopic image capture.

[0032] The region of interest setting unit 26 sets a plurality of regions of interest in the image of the subject 10 based on the positions of the skeleton of the subject 10 extracted by the skeleton detection unit 24 .

[0033] In embodiment 1, the region of interest setting unit 26 sets one rectangular region 51 that includes almost the entire image of the subject 10 as shown in Figure 5 based on the position of the skeleton of the subject 10, and sets a predetermined number of regions of interest 52 arranged vertically and horizontally by dividing the rectangular region 51 vertically and horizontally in a grid pattern as shown in Figure 6.

[0034] Specifically, region of interest setting unit 26 determines the coordinates of the left end, right end, top end, and bottom end of a rectangular region including subject 10 based on the coordinates of the skeletal feature points of subject 10, and sets rectangular region 51. For example, when the horizontal direction of rotated image 21a is the x-axis and the vertical direction is the y-axis, the coordinates a predetermined distance (e.g., 30 pixels) to the left of the coordinates of the feature point located at the leftmost position in the x-axis direction among the detected skeletal feature points (both ears, noses, both eyes, both shoulders, both elbows, both wrists, both hips, both knees, and both ankles) are set as the left end of the region, the coordinates a predetermined distance (e.g., 30 pixels) to the right of the coordinates of the feature point located at the rightmost position in the x-axis direction are set as the right end of the region, and the coordinates a predetermined distance (e.g., 30 pixels) below the coordinates of the feature point located at the bottommost position in the y-axis direction are set as the bottom end of the region. Furthermore, if the y-coordinate of the ear located on the lower side in the y-axis direction is Ye, and the y-coordinate of the shoulder located on the upper side in the y-axis direction is Ys, then the position of Ye-(Ys-Ye) is the upper end of the region.

[0035] 6, region of interest setting unit 26 divides set rectangular region 51 into a predetermined number of grid-shaped regions of interest. This sets a predetermined number of rectangular regions of interest 52 arranged vertically and horizontally. Here, a grid-shaped region of interest 52 of 10 vertical by 5 horizontal is set regardless of the size of rectangular region 51, but the number of regions of interest 52 to be set may be increased or decreased depending on the size of rectangular region 51.

[0036] On the other hand, the optical flow calculation unit 27 calculates the amount of movement (optical flow) of corresponding pixels between the most recent image 21a and the image 21a captured by the camera 21 at a timing a predetermined time before the timing of capturing the image 21a by performing a calculation. The optical flow is calculated using a known method such as the Farneback method.

[0037] The optical flow calculation unit 27 calculates the amount of movement (optical flow) of corresponding pixels between the most recently acquired image 21a and the image 21a acquired and rotated at a timing predetermined time before the most recently acquired image 21a, for each pixel (see Figure 7).

[0038] The determination unit 28 sets the region of interest 52 for an image in which pixel values ​​are the movement amounts (optical flows) calculated by the optical flow calculation unit 27 (see FIG. 8), and acquires the movement amounts of each pixel of the subject 10 in one region of interest 52. The determination unit 28 counts the number of pixels whose movement amounts are greater than a predetermined threshold. If the number of pixels whose movement amounts are greater than the threshold is greater than a predetermined value (e.g., 20% of the number of pixels in one region of interest 52), the region of interest is determined to have a movement amount greater than the threshold.

[0039] The determination unit 28 determines the amount of movement for all of the regions of interest 52 that have been set, and finds the number of regions of interest 52 whose amount of movement is greater than a threshold value.

[0040] If the number of regions of interest 52 determined to have a movement amount greater than the threshold is greater than a predetermined value (for example, three), the determining unit determines that a body movement has occurred in the subject 10.

[0041] A notification unit 29 is connected to the body movement detection unit 20. When the determination unit 28 determines that a body movement has occurred, the notification unit 29 notifies the user of this. For example, the notification unit 29 displays a predetermined notification dialog 161 notifying the user of the occurrence of a body movement on the display screen of the connected camera image display monitor 15 (see FIG. 10). The notification unit 29 may also display the amount of movement calculated by the optical flow calculation unit 27 as a numerical value or a graph on the display screen of the camera image display monitor 15 (see FIG. 11). Alternatively, the notification unit 29 may display both of these (see FIG. 10).

[0042] 12(a) and 12(b), the notification unit 29 may be configured to notify the presence of body movement by changing the color of the edge of the image 21a displayed on the camera image display monitor 15. For example, if there is no body movement, the notification unit 29 displays a green line on the edge of the image 21a, and if there is body movement, the notification unit 29 displays a red line on the edge of the image 21a.

[0043] The monitor that displays the notification is not limited to the camera image display monitor 15, but may be the fluoroscopic image display monitor 14 or the endoscopic image display monitor 16.

[0044] Furthermore, the notification unit 29 may notify the occurrence of body movement by an alarm sound or light.

[0045] The operation of each part of the body movement detection unit 20 will be described below with reference to the flow chart of FIG.

[0046] The functions of each section (22 to 26) of the body movement detection section 20 can be realized by software. In this case, the body movement detection section 20 is configured by a computer or the like equipped with a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory, and the CPU reads and executes programs stored in the memory to realize those functions. It is also possible to configure part or all of the body movement detection section 20 by hardware. For example, a circuit can be designed to realize the functions of each section using a custom IC such as an ASIC (Application Specific Integrated Circuit) or a programmable IC such as an FPGA (Field-Programmable Gate Array).

[0047] (Step S101) The image capturing unit 22 captures the images 21a at predetermined time intervals from the camera 21. The timing at which the image capturing unit 22 starts capturing images may be the timing at which a start instruction is received from the operator, or the timing at which the endoscopic image display monitor 16 starts displaying the endoscopic image.

[0048] (Step S102) The image rotation unit 23 rotates the image 21a by 90 degrees in a predetermined direction, so that the predetermined end of the tabletop 80 on which the head of the subject 10 is to be placed is positioned at the top of the image 21a.

[0049] (Step S103) The skeleton detection unit 24 extracts skeletal feature points (ears, nose, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc.) of all human figures in the image 21a and detects their two-dimensional coordinates (see FIG. 4). In this way, the skeletons of all human figures are extracted.

[0050] (Step S104) If the skeleton cannot be detected in step S103, the process returns to step S101 and the next image 21a is captured. If the skeleton can be detected, the process proceeds to step S105.

[0051] (Step S105) The subject skeleton extraction unit 25 calculates predetermined features using the position coordinates of the feature points of the skeletons of all human images detected by the skeleton detection unit 24, and extracts the skeleton of the subject 10 placed on the tabletop based on the calculated features.

[0052] In addition, the subject skeleton extraction unit 25 distinguishes the subject 10 from the extracted images of medical personnel around the tabletop 80, and extracts only the image of the subject 10, which is displayed on the camera image display monitor 15 as shown in Figures 1(b) and (c).

[0053] (Step S106) Based on the positions of skeletal feature points of the subject 10, the region of interest setting unit 26 sets one rectangular region 51 that includes almost the entire image of the subject 10 as shown in Fig. 5. The set rectangular region 51 is divided vertically and horizontally into a grid pattern as shown in Fig. 6, thereby setting a predetermined number of regions of interest 52 arranged vertically and horizontally.

[0054] (Step S107) The optical flow calculation unit 27 calculates, for each pixel, the amount of movement (optical flow) of corresponding pixels between the image 21a rotated in step S102 and the image 21a acquired and rotated at a timing a predetermined time before the image 21a acquired in step S102 (see FIG. 7). The image 21a used to calculate the amount of movement is not an image in which only the human image of the subject 10 is extracted, but the entire image including all human images.

[0055] (Step S108) The determination unit 28 determines whether or not an optical flow is detected in the area surrounding the subject's skeleton extracted in step S107. If an optical flow is detected in the area surrounding the subject's skeleton, the process proceeds to step S109. If an optical flow is not detected in the area surrounding the subject's skeleton, the process proceeds to step S112.

[0056] (Step S109) Determination unit 28 determines that a region of interest in which the number of pixels in which the amount of movement of pixels within region of interest 52 exceeds the threshold is greater than a predetermined value (for example, 20% of the number of pixels in one region of interest 52) ​​is a region of interest in which the amount of movement is greater than the threshold. Determination unit 28 performs this determination for all regions of interest 52. If there is a region of interest in which the amount of movement is greater than the threshold, proceed to step S110. If there is not a region of interest in which the amount of movement is greater than the threshold, return to step S101. If there is not a region of interest in which the amount of movement is greater than the threshold, proceed to step S112.

[0057] (Step S110) The determination unit 28 determines the number of regions of interest whose movement amount is greater than a threshold, and if the number is greater than a predetermined value (e.g., three), determines that a body movement has occurred in the subject 10, and proceeds to step S111. If the number is less than the predetermined value, proceeds to step S112.

[0058] (Step S111) When the determination unit 28 determines that a body movement has occurred, the notification unit 29 notifies the user that a body movement has occurred.

[0059] Specifically, the notification unit 29 notifies the user of the presence of body movement by displaying a predetermined notification dialog 161 on the camera image display monitor 15, as shown in Fig. 10. Alternatively, the notification unit 29 notifies the user of the presence of body movement by displaying the amount of movement calculated by the optical flow calculation unit 27 as a numerical value or a graph 162, as shown in Fig. 11. Alternatively, the notification unit 29 displays both the notification dialog 161 and the graph 162. Alternatively, the notification unit 29 may notify the user of the presence of body movement by changing the color of the border of the image 21a displayed on the camera image display monitor 15, as shown in Fig. 12.

[0060] Furthermore, the notification unit 29 may issue an alarm by sound or light when there is body movement.

[0061] (Steps S113 and S114) The notification unit 29 determines whether a predetermined time (e.g., 10 seconds) has elapsed since starting the body movement notification in step S111 (step S113), and if the predetermined time has elapsed, proceeds to step S114 to stop the notification and returns to step S101. On the other hand, in step S113, if the predetermined time has not elapsed since the body movement notification started, the notification is not stopped and the process returns to step S101.

[0062] (Step S112) Furthermore, if no optical flow is detected in the area surrounding the subject's skeleton in step S108, or if there is no area of ​​interest with a movement amount greater than the threshold in step S109, or if the number of areas of interest with a movement amount greater than the threshold is equal to or less than a predetermined value in step S110, then in step S112 it is determined whether or not body movement notification is currently being performed.

[0063] If a body movement notification is in progress, the process proceeds to step S113, where it is determined whether a predetermined time (for example, 10 seconds) has elapsed since the body movement notification started. If a body movement notification is not in progress, the process returns to step S101.

[0064] By performing steps S112 to S114 in this manner, the body movement notification can be continued for a predetermined period of time (for example, 10 seconds).

[0065] As described above, according to the first embodiment, an image of the subject 10 can be extracted and displayed, and further, it is possible to determine whether or not the subject 10 is moving, and if so, to notify the user of that fact.

[0066] The presence or absence of body movement is determined by dividing the rectangular region 51 covering the subject 10 in the image 21a into a predetermined number of regions of interest. This allows for more accurate body movement determination, with less influence from optical flow (movement amount) noise than when the determination is performed for the entire rectangular region 51. Optical flow noise is characterized in that pixels that are considered to have moved appear as small clusters across the entire image, whereas body movement is characterized in that, due to the movement of a part of the body, the moved pixels appear as large clusters in a portion of the image. Therefore, if there is a lot of noise, the number of pixels affected by noise may exceed a predetermined value (e.g., 20%) across the entire rectangular region 51. However, the number of pixels affected by noise within each region of interest divided into a grid rarely exceeds 20% of the pixels within the region of interest. Therefore, as in this embodiment, if the number of regions of interest with movement amounts greater than a predetermined threshold is greater than a predetermined value (e.g., three), body movement is determined to exist, allowing large clusters of moved pixels to be detected and less susceptible to noise.

[0067] Therefore, the X-ray imaging apparatus of this embodiment can accurately detect the body movement of the subject while capturing a fluoroscopic image. If body movement is detected, the doctor is notified, so that the doctor can easily grasp the body movement of the patient even during a procedure.

[0068] Specifically, because it is possible to detect bodily movements that require a doctor's attention at the initial stage (when the movement begins about 3 cm), it is possible to respond to the bodily movements early (holding down the subject's body, etc.). This makes it possible to prevent the subject from moving their hands to remove the endoscope or from trying to raise their body.

[0069] Furthermore, in step S105, the camera image display monitor 15 displays only the image of the subject 10, excluding images of medical personnel, which has the advantage that the doctor can easily understand the condition and body movements of the subject 10 by looking at the camera image display monitor 15.

[0070] <<Embodiment 2>> The X-ray imaging apparatus of the second embodiment will be described with reference to FIGS.

[0071] When performing a procedure of inserting an endoscope into a subject 10 while taking fluoroscopic images using the X-ray imaging device 1, many medical professionals are present around the subject 10, operating the endoscope and moving the endoscope itself, as shown in Fig. 2. Therefore, when a rectangular region 51 that covers almost the entire subject 10 is set and regions of interest 52 are set by dividing the rectangular region 51, as in the first embodiment, some regions of interest 52a may be set so as to overlap images of the doctor's hands, the optical cable of the endoscope, etc., as shown in Fig. 13.

[0072] In such a case, the determination unit 28 determines the amount of movement (optical flow) of the region of interest 52a at that position based on the movement of the doctor operating the endoscope, the movement of the endoscope itself, or the movement of the doctor or other medical professional, and therefore erroneously detects the region of interest as having a movement amount greater than the threshold.

[0073] Therefore, the region of interest setting unit 26 of the X-ray imaging apparatus 1 of the second embodiment executes step S506 of the flow of Fig. 15 instead of step S106 of the flow of Fig. 9 of the first embodiment. In step S506, when setting the region of interest, the region of interest setting unit 26 partially sets regions of interest 71 and 72 based on the positions of predetermined feature points among the skeletal feature points of the subject 10.

[0074] Specifically, in step S506, region of interest setting unit 26 sets a region of interest 71 of a predetermined size centered on the shoulders, elbows, wrists, hips, knees, and ankles, among the skeletal feature points of subject 10 detected by skeleton detection unit 24. Furthermore, region of interest setting unit 26 determines the position of the head as follows, and sets a region of interest 72 of a predetermined size centered on the head position. To determine the position of the head, region of interest setting unit 26 first draws line 75 that passes through the coordinates of ear 74 of subject 10, one of the two ears detected by skeleton detection unit 24, that is closer to the center of subject 10's body in the y-axis direction (the longitudinal direction of tabletop 80), and is parallel to the x-axis direction (the lateral direction of tabletop 80). Furthermore, region of interest setting unit 26 draws line 76 that passes through midpoint 41 of the line connecting the shoulders and is parallel to the y-axis direction. Region of interest setting unit 26 sets the intersection of line 75 and line 76 as head position 77 and sets region of interest 72 with head position 77 as the center.

[0075] The regions of interest 71 and 72 have the same predetermined size, for example, a size slightly larger than the arm circumference of a 170 cm adult male, or a predetermined size that does not overlap with the operator's hand.

[0076] Other steps in the flow of Fig. 15 are performed in the same manner as in the flow of Fig. 9 of embodiment 1. The configuration of the X-ray imaging device 1 is also the same as in embodiment 1, so a description thereof will be omitted.

[0077] The determination unit 28 determines that a region of interest in which the number of pixels in the regions of interest 71, 72 whose movement amount exceeds the threshold is greater than a predetermined value (e.g., 20% of the number of pixels in one region of interest) is greater than the threshold. The determination unit 28 calculates the number of regions of interest in which the movement amount is greater than the threshold, and if the number is greater than a predetermined value (e.g., 3), determines that a body movement has occurred in the subject 10.

[0078] As described above, in the second embodiment, the region of interest setting unit 26 sets the regions of interest 71, 72 only at joint positions that are easily moved (both shoulders, both elbows, both wrists, both hips, both knees, and both ankles) and at the position of the head based on the skeletal feature points of the subject 10, thereby making it possible to set the regions of interest 71, 72 while avoiding the endoscope cable and the doctor's hands, as shown in Fig. 17. This reduces the possibility of erroneously determining the movement of the doctor's hands or the movement of the endoscope cable as body movement of the subject 10.

[0079] 15, when the notification unit 29 displays on the camera image display monitor 15 that a body movement has occurred, it is also possible to use a display method such as that shown in Fig. 16 in addition to the display methods shown in Figs. 10 to 12 described in embodiment 1. In addition to the display methods shown in Figs. 10 to 12, the display method shown in Fig. 16 can also be adopted.

[0080] 16(a) and 16(b), frames indicating the regions of interest 71, 72 on the subject 10 used for body movement detection are superimposed on the image 21a of the subject 10 displayed on the camera image display monitor 15 (see FIG. 16(a)), so that it is possible to see which regions of interest are moving. For example, the region of interest 71a determined in step S109 to have a movement amount greater than a threshold is displayed in color (FIG. 16(b)). Alternatively, the region of interest 71b determined in step S109 to have a movement amount greater than the threshold is displayed in a color different from the other regions of interest. By looking at the image 21a of the subject 10 displayed on the camera image display monitor 15, the doctor can easily see which part of the body is moving.

[0081] Furthermore, the method for calculating the head position 77 is not limited to the above method, and any method that can calculate a point inside the image of the head may be used. For example, the midpoint of the line segment connecting both ears can be set as the head position.

[0082] <<Embodiment 3>> The X-ray imaging apparatus of the third embodiment will be described with reference to FIG.

[0083] 15, the X-ray imaging apparatus of the third embodiment is configured to set a region of interest 71 at a feature point of the subject 10, and further set a region of interest at a head position 77 of the subject 10. In the third embodiment, when setting a region of interest 78 at the head position 77 of the subject 10, the region of interest setting unit 26 sets the region of interest 78 at a position slightly shifted from the region of interest 72 set in the second embodiment. In other words, the region of interest 78 is set so that a part of the outline of the head of the subject 10 excluding the face is positioned inside the region of interest 78.

[0084] This will be explained in more detail. The orientation of the face of subject 10 on which side of tabletop 80 it faces is set in advance in order to insert an endoscope. Region of interest setting unit 26 sets region of interest 78 so that, for example, of two sides of region of interest 78 parallel to the y-axis direction (the long axis direction of tabletop 80), the side closer to the face of subject 10 is located above head position 77. This makes it possible to set region of interest 78 so that part of the contour of the occipital region of subject 10 is located inside region of interest 78.

[0085] In this way, by setting the region of interest 78 so that part of the head contour excluding the face at head position 77 of subject 10 is located inside the region of interest 78, the region of interest 78 can be set at a position that avoids the endoscope inserted through the face and the doctor's hands.

[0086] Therefore, when the determining unit 28 determines the body movement of the subject 10, it is possible to more reliably avoid the influence of the movement of the endoscope or the movement of the doctor's hand than in the second embodiment.

[0087] The method for calculating the head position 77 is as explained in the second embodiment. The display method is also the same as in the second embodiment.

[0088] Furthermore, the operation and device configuration other than the setting position of the region of interest 78 are the same as those in the first embodiment, and therefore a description thereof will be omitted. [Explanation of symbols]

[0089] 1 X-ray imaging device 10 Subject 11 X-ray irradiation device 12 X-ray detector 13 X-ray image generation section 14 Fluoroscopic image display monitor 15 Camera image display monitor 16 Endoscopic image display monitor 20 Body movement detection unit 21 Camera 21a Image 22 Image capture section 23 Image rotation unit 24 Skeleton detection unit 25. Subject skeleton extraction unit 26 Region of interest setting section 27 Optical flow calculation unit 28 Judgment section 29 Notification Department 41 Midpoint of the line connecting both shoulders 45 Center of image 50 X-ray support 51 Rectangular area 52 Areas of Interest 52a Area of ​​Interest 60 X-ray vertical movement mechanism 61 X-ray tilt mechanism 62 Pressing tool 70 Stand 71 Areas of Interest 72 Areas of Interest 74 Ears 75 Lines parallel to the x-axis 76 Lines parallel to the y-axis 77 Head Position 80 Top Plate 81 Top plate support 161 Notification Dialogs 162 graphs

Claims

1. an X-ray detector that detects the X-rays that have been irradiated from the X-ray irradiator and passed through the subject; and a body movement detection unit that receives an image from a camera that captures an image of the subject, processes the image, and detects the occurrence of body movement of the subject, the body movement detection unit includes a skeleton detection unit, an optical flow calculation unit, a region of interest setting unit, and a determination unit; the skeleton detection unit detects a skeleton of a human figure included in the image, and extracts a skeleton of the subject placed on the tabletop based on the detected skeleton; the optical flow calculation unit calculates a movement amount of corresponding pixels between the image and an image acquired by the camera at a timing a predetermined time before the timing of acquisition of the image; the region of interest setting unit sets a plurality of regions of interest in the image of the subject based on the positions of the subject's skeleton extracted by the skeleton detection unit; the determination unit determines the number of regions of interest in which the number of pixels in the region of interest, where the amount of movement of the subject's pixels is greater than a predetermined threshold, is greater than a predetermined value, and determines that body movement has occurred in the subject if the determined number of regions of interest is greater than the predetermined value.

2. 2. The X-ray imaging device according to claim 1, The skeleton detection unit detects the skeletons of all human images included in the image, calculates predetermined features based on the detected skeletons, and extracts the skeleton of the subject placed on the tabletop based on the calculated features.

3. 2. The X-ray imaging device according to claim 1, wherein the region of interest setting unit sets a single rectangular region that includes the entire image of the subject based on the position of the subject's skeleton, and sets a plurality of the regions of interest that are arranged vertically and horizontally by dividing the rectangular region vertically and horizontally into a grid pattern.

4. 2. The X-ray imaging apparatus according to claim 1, wherein the region of interest setting unit determines positions of a plurality of predetermined feature points in a skeleton of the subject, and sets the region of interest using the determined plurality of feature points.

5. 5. The X-ray imaging apparatus according to claim 4, wherein the region of interest setting unit sets a region of interest at each of the positions of a plurality of the feature points.

6. 5. The X-ray imaging apparatus according to claim 4, wherein when the predetermined feature point is a feature point of the subject's head, the region of interest setting unit sets the region of interest so that a contour of the head is positioned inside the region of interest.

7. 2. The X-ray imaging apparatus according to claim 1, wherein the body movement detection unit further comprises a notification unit that notifies a user of the presence of body movement when the determination unit determines that body movement has occurred.

8. 8. The X-ray imaging device according to claim 7, wherein the notification unit performs at least one of displaying a predetermined notification dialog on a connected display screen and displaying the amount of movement calculated by the optical flow calculation unit as a numerical value or a graph.

9. 8. The X-ray imaging apparatus according to claim 7, wherein the notification unit issues the notification by sound.

10. A method for processing an image captured by a camera of an X-ray imaging device provided with the camera, comprising: detecting a skeleton of a human figure included in the image, and extracting a skeleton of a subject placed on a tabletop of the X-ray imaging device based on the detected skeleton; calculating a movement amount of corresponding pixels between the image and an image acquired by the camera at a timing a predetermined time before the timing of acquisition of the image; setting a plurality of regions of interest in the image of the subject based on the position of the bone structure of the subject; The number of regions of interest in which the amount of movement of the subject's pixels within the region of interest is greater than a predetermined threshold is calculated, and if the calculated number of regions of interest is greater than the predetermined value, it is determined that a body movement has occurred in the subject.

10. An image processing method for an X-ray imaging device equipped with a camera, comprising:

Images