X-ray imaging device and image processing device
The X-ray imaging device enhances IVR by isolating patient images using skeletal detection, allowing continuous monitoring without gaze shifts, thus improving situational awareness and reducing distractions in IVR procedures.
Patent Information
- Application Number
- JP2023086661
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-12-06
AI Technical Summary
In IVR procedures, doctors need to frequently shift their gaze between patient monitors and the patient, making it difficult to maintain focus on the patient's condition due to the inclusion of unnecessary personnel in surveillance camera views and the need to visually check the patient's face, feet, or hands, which can obscure the patient's situation.
An X-ray imaging device with a top plate and integrated surveillance camera system that processes images to extract and display the patient's image on a separate monitor, using skeletal detection algorithms to isolate the patient from other figures, allowing continuous monitoring without shifting gaze.
Enables continuous patient monitoring by displaying only the patient's image on a dedicated monitor, improving situational awareness and reducing distractions from other personnel, while maintaining focus on fluoroscopic and endoscopic images.
Smart Images

Figure 2024169850000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an X-ray imaging device that irradiates an object with X-rays to obtain an image. [Background technology]
[0002] IVR (Interventional Radiology) is a known technique in which an endoscope or catheter is inserted into a patient and various procedures are performed on the patient while irradiating the patient with X-rays from an X-ray imaging device to obtain fluoroscopic images. A doctor performs procedures on the patient while looking at a monitor displaying a fluoroscopic image and a monitor displaying an endoscopic image.
[0003] When checking the patient's condition while performing a procedure while viewing fluoroscopic images, the doctor moves his or her gaze to the patient's face, feet, or hands to check visually, or moves his or her gaze to a monitor displaying images from a surveillance camera attached to the ceiling or upper part of a wall in the room and checks the patient via the surveillance camera image.
[0004] On the other hand, Patent Document 1 discloses a technique for detecting the body movement of a subject by performing image processing on a fluoroscopic image.
[0005] Patent Document 2 discloses a technology for identifying a worker performing a specific task from among multiple workers by photographing the workers with a camera and processing the images to detect their skeletons. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2021-186199 A [Patent Document 2] Patent Publication No. 2021-196783 Summary of the Invention [Problem to be solved by the invention]
[0007] In IVR, the patient is generally placed next to the doctor, and the monitors displaying fluoroscopic and endoscopic images are generally placed in front of the doctor. This means that the doctor must move his or her gaze from the monitor to the patient in order to visually check the patient's condition.
[0008] On the other hand, when a doctor checks on a patient's condition by looking at a monitor showing surveillance camera images in the room, it is difficult to instantly grasp the patient's condition because the surveillance camera images show not only the patient, but also the doctor, technicians, and nurses.
[0009] It is also possible to limit the area in which the surveillance camera image is displayed so that nothing other than the patient is displayed on the monitor, but if the position or orientation of the tabletop on which the patient is placed is changed as the procedure progresses, the subject may no longer be visible on the surveillance camera.
[0010] An object of the present invention is to extract and display an image of a subject on a tabletop on which a fluoroscopic image is being captured from an image captured by a surveillance camera. [Means for solving the problem]
[0011] In order to achieve the above object, the present invention provides an X-ray imaging device having a tabletop on which a subject is placed, an X-ray irradiator that irradiates the subject with X-rays, an X-ray detector that detects the X-rays irradiated from the X-ray irradiator and transmitted through the subject, and a camera image processor that receives an image from a camera that images the subject and processes the image. The image processor detects the skeletons of multiple human figures included in the image, extracts an image of the subject placed on the tabletop from the multiple human figures based on the detected skeletons, and displays the image on a connected display device. Effect of the Invention
[0012] According to the present invention, an image of the patient on the tabletop on which the fluoroscopic image is being taken can be extracted from the image captured by the surveillance camera and displayed on a monitor, allowing the doctor to visually check the patient's condition by looking at the surveillance camera image monitor without having to look down at the patient. [Brief description of the drawings]
[0013] [Figure 1] FIG. 1A is a front view of an X-ray imaging device 1 according to a first embodiment of the present invention, and FIGS. 1B and 1C are diagrams showing images displayed on a monitoring image display monitor. [Diagram 2] FIG. 2 is a functional block diagram of the X-ray imaging apparatus according to the first embodiment. [Diagram 3] 3A to 3C are diagrams showing an image captured by a monitoring camera of the X-ray imaging device of the first embodiment and a skeleton detected from the image. [Figure 4] 3 is an image diagram in which a region around the skeleton of a subject 10 is extracted from an image captured by a surveillance camera of the X-ray imaging device of the first embodiment. FIG. [Diagram 5] 4 is a flowchart showing the operation of the X-ray imaging apparatus of the first embodiment. [Figure 6] 10 is a flowchart showing the operation of the X-ray imaging apparatus according to the second embodiment. [Figure 7] 13A to 13C are diagrams showing an image captured by a monitoring camera of the X-ray imaging device of the second embodiment and a skeleton detected from the image. [Figure 8] 11 is a flowchart showing the operation of the X-ray imaging apparatus according to the third embodiment. [Figure 9] 13A to 13C are diagrams showing an image captured by a monitoring camera of the X-ray imaging device of the third embodiment and a skeleton detected from the image. [Figure 10] FIG. 11 is a functional block diagram of an X-ray imaging apparatus according to a fourth embodiment. [Figure 11] 10 is a flowchart showing the operation of the X-ray imaging apparatus according to the fourth embodiment. [Figure 12] 13A to 13C are views showing an image captured by a monitoring camera of the X-ray imaging device of the fourth embodiment and the optical flow detected from the image. [Figure 13] FIG. 13 is a functional block diagram of an X-ray imaging apparatus according to a fifth embodiment. [Figure 14]13 is a flowchart showing the operation of the X-ray imaging apparatus according to the fifth embodiment. [Figure 15] 13A to 13C are views showing an image captured by a monitoring camera of the X-ray imaging device of the fifth embodiment and the optical flow detected from the image. [Figure 16] 13A is a diagram of a monitor screen displaying an image captured by a surveillance camera of the X-ray imaging device of embodiment 5, and FIG. 13B is a diagram of the monitor screen displaying an enlarged view of the face area of the image of the subject. [Figure 17] FIG. 13 is a functional block diagram of an X-ray imaging apparatus according to a sixth embodiment. [Figure 18] 13 is a flowchart showing the operation of the X-ray imaging apparatus of the sixth embodiment. [Figure 19] FIG. 13 is a functional block diagram of an X-ray imaging apparatus according to a seventh embodiment. [Figure 20] 13 is a flowchart showing the operation of the X-ray imaging apparatus of the seventh embodiment. [Figure 21] 13 is a diagram showing an image captured by a monitoring camera of the X-ray imaging device of embodiment 7, and a skeleton and an X-ray irradiated area detected from the image. [Figure 22] FIG. 13 is a functional block diagram of an X-ray imaging apparatus according to an eighth embodiment. [Figure 23] 20 is a flowchart showing the operation of the X-ray imaging apparatus of the eighth embodiment. [Figure 24] 13 is a diagram showing an image captured by a monitoring camera of the X-ray imaging device of embodiment 8, and a skeleton and an X-ray irradiated area detected from the image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, an X-ray irradiation device according to an embodiment of the present invention will be described with reference to the drawings.
[0015] <Embodiment 1> The X-ray imaging device 1 of the first embodiment will be described with reference to Figs. 1 to 3. Fig. 1(a) is a front view of the X-ray imaging device 1, and Figs. 1(b) and 1(c) are images displayed on a monitoring image display monitor. Fig. 2 is a functional block diagram of the X-ray imaging device 1. Fig. 3 is a diagram showing an image captured by a monitoring camera and a skeleton detected from the image.
[0016] As shown in FIG. 1(a), the X-ray imaging device 1 includes a tabletop 80 on which a subject 10 is placed, and an X-ray irradiator 11 that irradiates the subject 10 with X-rays. The X-ray irradiator 11 is supported on a stand 70 by an X-ray support unit 50. The stand 70 includes an X-ray up-down movement mechanism 60 that moves the X-ray support unit 50 up and down, a longitudinal direction movement mechanism (not shown) that moves the X-ray support unit 50 in the longitudinal direction of the tabletop 80, a transverse direction movement mechanism (not shown) that moves the X-ray support unit 50 in the transverse direction of the tabletop 80, and an X-ray tilt mechanism 61 that tilts the central axis of the X-ray support unit 50 about the transverse direction of the tabletop. The X-ray support unit 50 includes an X-ray rotation mechanism (not shown) that rotates the X-ray irradiator 11 about the longitudinal direction of the tabletop. The X-ray support part 50 is provided with a pressing tool 62 used when pressing the subject 10 .
[0017] On the other hand, the top plate 80 is supported on the stand 70 by a top plate support portion 81 .
[0018] The flat panel X-ray detector 12 is disposed inside the top board support part 81. The flat panel X-ray detector 12 is a flat detector in which X-ray detection elements are arranged two-dimensionally. The X-ray detector 12 detects X-rays irradiated from the X-ray irradiator 11 and transmitted through the subject 10.
[0019] The X-ray support part 50 or the X-ray irradiator 11 is provided with a surveillance camera 21 that images the subject 10. The surveillance camera 21 is provided at a position that does not block the X-rays irradiated from the X-ray irradiator 11. The angle of view of the surveillance camera 21 is set so that the entire body of the subject 10 on the tabletop 80 is imaged as shown in FIG. 1(b). The surveillance camera 21 may be fixed to a cover (housing) of the X-ray irradiator 11.
[0020] As shown in Fig. 2, an X-ray image generating unit 13 is connected to the flat panel X-ray detector 12. The X-ray image generating unit 13 receives signals output by each X-ray detection element of the X-ray detector 12 upon detecting X-rays, and generates an X-ray image (here, a fluoroscopic image) at a predetermined frame rate. A fluoroscopic image display monitor 14 is connected to the X-ray image generating unit 13. The fluoroscopic image is displayed on the fluoroscopic image display monitor 14.
[0021] Meanwhile, a camera image processing unit 20 is connected to the surveillance camera 21. The camera image processing unit 20 receives an image from the surveillance camera 21 and processes the image. Specifically, the camera image processing unit 20 receives an image from the surveillance camera 21, detects the skeletons of multiple human figures contained in the received image, and extracts an image of the subject 10 sitting on the top board 80 from the multiple human figures based on the detected skeletons. The camera image processing unit 20 displays the extracted image of the subject 10 on the connected surveillance image display monitor 15.
[0022] The camera image processing unit 20 includes an image capture unit 22, an image rotation unit 23, a subject's skeletal structure determination unit 25, and a subject's region extraction unit .
[0023] The image capture unit 22 captures the images 21a output by the surveillance camera 21 at a predetermined time interval. The predetermined time interval can be set to be approximately the same as the frame rate of the fluoroscopic images generated by the X-ray image generation unit 13.
[0024] The image 21a is an image captured so that the body axis of the subject on the tabletop 80 corresponds to the horizontal axis of the image.
[0025] The image rotation unit 23 rotates the image 21a by 90 degrees in a predetermined direction so that the head position of the image of the subject 10 on the tabletop 80 is at the top of the image 21a as shown in Fig. 3. Since it is predetermined which direction on the tabletop 80 the subject 10 should be mounted so that his head is oriented, the direction of rotation can be determined in advance based on the predetermined head position on the tabletop 80 and the orientation of the surveillance camera 21.
[0026] The skeleton detection unit 24 extracts all skeletal feature points (ears, nose, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc.) shown in the image 21a as shown in FIG. 3. The skeleton detection unit 24 detects all skeletons by detecting their two-dimensional coordinates. The skeleton detection unit 24 can be realized by a known posture detection algorithm. For example, the posture estimation algorithm Open Pose (registered trademark) included in Open VINO (registered trademark) provided by Intel Corp can be used.
[0027] The subject skeleton determination unit 25 determines all people whose ears, both shoulders, and both hips are detected from the skeleton feature points detected by the skeleton detection unit 24 as subject candidates 40. A person whose ears, both shoulders, and both hips are detected is recognized as a subject candidate 40 even if the person is standing (e.g., the operator 30).
[0028] The subject skeleton determining unit 25 calculates the skeleton center line for each subject candidate. Specifically, a line connecting the midpoints of both shoulders and the midpoints of both waists is calculated, and this calculated line is set as a skeleton center line 135. The subject skeleton determining unit 25 calculates an angle θ between the calculated skeleton center line 135 and a vertical center line 136 of the image 21a rotated by the image rotating unit 23. The subject skeleton determining unit 25 compares the angles θ of a plurality of subject candidates 40, and determines the subject candidate 40 with the smallest angle θ. In other words, the subject 10 is determined to be the subject whose skeleton center line 135 is closest to the center line 136 of the image 21a.
[0029] The subject region extraction unit 26 extracts the skeleton determined to be the subject 10 and the surrounding region in the image 21a as shown in FIG. 4. Specifically, based on the coordinates of the feature points of the skeleton of the subject 10, the coordinates of the left end, right end, top end and bottom end of the rectangular region including the subject 10 are determined, and the rectangular region is extracted. For example, when the horizontal direction of the rotated image 21a is the x-axis and the vertical direction is the y-axis, the coordinates a predetermined distance (e.g., 30 pixels) to the left of the coordinates of the feature point located at the leftmost position in the x-axis direction among the detected skeletal feature points (both ears, noses, both eyes, both shoulders, both elbows, both wrists, both hips, both knees, both ankles) are set as the left end of the region, the coordinates a predetermined distance (e.g., 30 pixels) to the right of the coordinates of the feature point located at the rightmost position in the x-axis direction are set as the right end of the region, and the coordinates a predetermined distance (e.g., 30 pixels) below the coordinates of the feature point located at the bottom in the y-axis direction are set as the bottom end of the region. In addition, if the y coordinate of the ear located on the lower side in the y-axis direction is Ye and the y coordinate of the shoulder located on the upper side in the y-axis direction is Ys, the position of Ye-(Ys-Ye) is the upper end of the region. Subject region extraction unit 26 extracts a rectangular region, rotates the image of the extracted region of subject 10 by 90 degrees in a predetermined direction, and then displays it on monitoring image display monitor 15. The direction in which subject region extraction unit 26 rotates the image of the region of subject 10 by 90 degrees is opposite to the direction in which image rotation unit 23 rotates image 21a.
[0030] Hereinafter, the operation of each part of the camera image processing unit 20 will be described with reference to the flow of FIG.
[0031] The functions of each unit (22-26) of the camera image processing unit 20 can be realized by software. In this case, the camera image processing unit 20 is configured by a computer or the like equipped with a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory, and the CPU reads and executes a program stored in the memory to realize those functions. It is also possible to configure a part or the whole of the camera image processing unit 20 by hardware. For example, a circuit can be designed to realize the functions of each unit using a custom IC such as an ASIC (Application Specific Integrated Circuit) or a programmable IC such as an FPGA (Field-Programmable Gate Array).
[0032] (Step 101) The image capturing unit 22 captures the image 21a from the surveillance camera 21 at a predetermined time interval (see FIG. 1(b)). The image capturing unit 22 may start capturing the image when it receives a start instruction from the operator, or when the endoscopic image display monitor 16 starts displaying the endoscopic image.
[0033] (Step 102) The image rotation unit 23 rotates the image 21a by 90 degrees in a predetermined direction, so that the end of the tabletop 80 on which the head of the subject 10 is to be placed is positioned at the top of the image 21a.
[0034] (Step 103) The skeleton detection unit 24 extracts all skeletal feature points (ears, nose, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc.) appearing in the image 21a and detects their two-dimensional coordinates (see FIG. 3).
[0035] (Steps 104 and 105) The subject skeleton determining section 25 determines whether there is a person whose ears, both shoulders, and both hips are detected from the skeleton detected by the skeleton detecting section 24, and recognizes the person as a subject candidate 40.
[0036] (Step 106) The subject skeleton determination unit 25 calculates a skeleton center line 135 connecting the midpoints of both shoulders and both waists for the subject candidate 40 identified in step 105 (see FIG. 3).
[0037] (Step 107) Subject skeleton determination section 25 calculates angle θ between calculated skeleton center line 135 and vertical center line 136 of image 21a rotated in step 102 (see FIG. 3).
[0038] (Steps 108 and 109) The subject skeletal determination unit 25 compares the angle θ of the subject candidate 40 calculated in step 107 with the angle θ of the subject candidate stored in the built-in memory 25a. If the angle θ calculated in step 107 is smaller than the angle θ in the memory 25a, the angle θ calculated in step 107 is overwritten and saved (updated) over the angle θ in the memory 25a, and the coordinates of the feature point of the subject candidate 40 whose angle θ was calculated in step 107 are overwritten and updated over the subject candidate in the memory 25a. Note that the angle θ of the first subject candidate 40 is not stored in the memory 25a, so the angle θ calculated in step 107 and the subject candidate 40 are stored in the memory 25a without comparison.
[0039] (Steps 110 and 111) The subject skeleton determination unit 25 determines whether any of the skeletons detected in step 103 have not been subjected to the determination in step 104, and if any skeletons have not been subjected to the determination in step 104, the process returns to step 104 to determine whether the ears, both shoulders, and both hips have been detected, and steps 105 to 109 are repeated. On the other hand, if, in step 110, there are no skeletons that have not been subjected to the determination in step 104, the process proceeds to step 111 and an image of the area surrounding the skeleton of the subject 10 currently stored in memory 25a is extracted (see FIG. 4).
[0040] (Step 112) Subject skeletal determination section 25 rotates the image of the region of subject 10 extracted in step 111 by 90 degrees in the opposite direction to that in step 102, and displays it on monitoring image display monitor 15 (see FIG. 1(c)).
[0041] In the X-ray imaging apparatus of the first embodiment, by disposing the monitoring image display monitor 15 alongside the fluoroscopic image display monitor 14 and the endoscopic image display monitor 16 in IVR (Interventional Radiology), the doctor can visually confirm the condition of the subject 10 by only slightly moving his / her line of sight to the monitoring image display monitor 15 located right next to it while observing the fluoroscopic image and the endoscopic image on the monitors 14, 16. This allows the doctor to visually confirm the subject 10 without dropping his / her line of sight from the monitors 14, 16 to the subject 10 on the tabletop 80.
[0042] Since the image displayed on the monitoring image display monitor 15 is an image of the skeleton of the subject 10 and the surrounding area, even if a doctor, technician, or nurse other than the subject 10 is captured in the monitoring camera image, only the subject 10 can be confirmed on the monitoring image display monitor 15. Therefore, the doctor can instantly grasp the patient's condition by looking at the monitoring image display monitor 15.
[0043] In addition, since the monitoring camera 21 is provided on the X-ray support part 50 or the X-ray irradiator 11, even if the operator moves the position or orientation of the X-ray irradiator 11 as the procedure progresses, the monitoring camera 21 follows the X-ray irradiator 11. Therefore, the subject 10 can be constantly photographed by the monitoring camera 21.
[0044] <Embodiment 2> An X-ray imaging apparatus according to the second embodiment will be described.
[0045] The X-ray imaging device of embodiment 1 is configured to determine that the skeleton having the smallest angle between the skeleton center line 135 and the image center line 136 in the image 21a is the subject 10, while the X-ray imaging device of embodiment 2 determines that the skeleton having the largest shoulder width (for example) on the image 21a is the subject 10.
[0046] As a result, in the image processing device of the second embodiment, the amount of calculation in the camera image processing unit 20 is smaller than that in the first embodiment, and processing speed is faster.
[0047] The configuration of the X-ray imaging apparatus of the second embodiment is similar to that of the first embodiment, but the operation of the subject skeletal determination unit 25 of the camera image processing unit 20 is different from that of the first embodiment. The operation of the camera image processing unit 20 will be described below with reference to the flow of FIG. 6, focusing on the points different from the first embodiment.
[0048] (Steps 101-103) An image capture unit 22 captures an image from the surveillance camera 21, and an image rotation unit 23 rotates the image 21a by 90 degrees. A skeleton detection unit 24 extracts all skeleton feature points appearing in the image 21a.
[0049] (Steps 204 and 205) The subject skeleton determination unit 25 determines whether there is a person whose ears and both shoulders are detected from the skeleton detected by the skeleton detection unit 24, and recognizes the person as a subject candidate 40. In the second embodiment, unlike the first embodiment, the detection of both hips is not a requirement for determination.
[0050] (Step 206) The subject's skeleton determining section 25 calculates the distance between the shoulders (shoulder width) W for the subject candidate 40 identified in step 205 (see FIG. 7).
[0051] (Steps 207 and 208) The subject skeleton determination unit 25 compares the shoulder width W of the subject candidate 40 calculated in step 206 with the shoulder width W of the subject candidate stored in the built-in memory 25a. If the shoulder width W calculated in step 206 is larger than the shoulder width W in the memory 25a, the shoulder width W calculated in step 206 is overwritten and saved (updated) on the shoulder width W in the memory 25a, and the coordinates of the feature point of the subject candidate 40 whose shoulder width W was calculated in step 206 are overwritten and saved on the subject candidate in the memory 25a to update it. Note that the shoulder width W of the first subject candidate 40 is not stored in the memory 25a, so the shoulder width W calculated in step 206 and the subject candidate 40 are stored in the memory 25a without comparison.
[0052] (Steps 110~112) The subject skeleton determination unit 25 determines whether there is any skeleton that has not been determined in step 204 among the skeletons detected in step 103, and if there is any skeleton that has not been determined, the process returns to step 204. On the other hand, if there is no skeleton that has not been determined in step 110, the process proceeds to step 111, where an image of the area around the skeleton of the subject 10 currently stored in the memory 25a is extracted, rotated 90 degrees in a predetermined direction, and displayed on the monitoring image display monitor 15.
[0053] The X-ray imaging apparatus of the second embodiment determines whether a candidate subject 40 is a subject 10 based on the body size on the image 21a of the candidate subject 40, and therefore can determine the subject with a smaller amount of calculation than the first embodiment.
[0054] Furthermore, since the X-ray imaging device of embodiment 2 does not use the waist bone structure to identify the subject 10, it is possible to identify the subject 10 even if the subject's 10 waist is covered with cloth or the like during IVR (Interventional Radiology) and is not photographed by the surveillance camera 21.
[0055] The other configurations and effects are similar to those of the first embodiment, and therefore the description thereof will be omitted.
[0056] <Embodiment 3> An X-ray imaging apparatus according to a third embodiment will be described.
[0057] The X-ray imaging device of embodiment 1 determines the subject 10 based on the angle between the skeleton center line 135 and the image center line 136 in the image 21a, so it is difficult to detect the subject 10 when the subject 10 and the operator or doctor are facing in the same direction.
[0058] On the other hand, in the second embodiment, the subject is identified based on the size of the skeleton in the image 21a, so that it is difficult to detect a subject 10 with a small body, such as a child.
[0059] Therefore, the X-ray imaging apparatus of the third embodiment determines the subject based on the distance from the center of the image 21a to the bone structure and the angle of the line connecting both shoulders.
[0060] The configuration of the X-ray imaging apparatus of the third embodiment is similar to that of the first embodiment, but the operation of the subject skeletal determination unit 25 of the camera image processing unit 20 is different from that of the first embodiment. The operation of the camera image processing unit 20 will be described below with reference to the flow of FIG. 8, focusing on the differences from the first embodiment.
[0061] (Steps 101-103) An image capture unit 22 captures an image from the surveillance camera 21, and an image rotation unit 23 rotates the image 21a by 90 degrees. A skeleton detection unit 24 extracts all skeleton feature points appearing in the image 21a.
[0062] (Steps 304 and 305) The subject skeleton determining section 25 determines whether there is a person whose ears and both shoulders are detected from the skeleton detected by the skeleton detecting section 24, and recognizes the person as a subject candidate 40.
[0063] (Step 306) Subject skeleton determination unit 25 calculates, for subject candidate 40 identified in step 305, distance L between midpoint Os of the line connecting both shoulders and center Oi of image 21a rotated in step 102 (see FIG. 9).
[0064] (Step 307) The subject skeleton determination unit 25 calculates an angle δ between a line 45 connecting both shoulders and the horizontal direction of the image for the subject candidate 40 identified in step 305 (see FIG. 9).
[0065] (Steps 308 and 309) The subject skeletal determination unit 25 determines whether the distance L calculated in step 306 and the angle δ calculated in step 308 are smaller than the distance L and angle δ of the subject candidate stored in the built-in memory 25a. If they are smaller, the distance L and angle δ calculated in steps 306 and 307 are overwritten and saved (updated) on the distance L and angle δ in the memory 25a, and the coordinates of the feature point of the subject candidate 40 for which the distance L and angle δ were calculated in steps 306 and 307 are overwritten and saved on the subject candidate in the memory 25a for updating.
[0066] Here, the image of a person whose distance L and angle δ are both smaller than those of the other subject candidates is determined to be the image of the subject 10, but the image of a person whose one of the distance L and angle δ is smaller than those of the other subject candidates may also be determined to be the image of the subject 10.
[0067] (Steps 110~112) The subject skeleton determination unit 25 determines whether there is any skeleton that has not been determined in step 204 among the skeletons detected in step 103, and if there is any skeleton that has not been determined, the process returns to step 204. On the other hand, if there is no skeleton that has not been determined in step 110, the process proceeds to step 111, where an image of the area around the skeleton of the subject 10 currently stored in the memory 25a is extracted, rotated 90 degrees in a predetermined direction, and displayed on the monitoring image display monitor 15.
[0068] As a result, even if the operator or doctor is facing the same direction as the subject 10 or even if the subject 10 is small, the subject 10 can be detected with high accuracy, an image can be extracted, and displayed on the monitoring image display monitor 15.
[0069] <Embodiment 4> An X-ray imaging apparatus according to a fourth embodiment will be described.
[0070] The X-ray imaging apparatus of the fourth embodiment detects wearoff of the anesthesia of the subject 10 by further processing the image of the subject 10 extracted by any one of the first to third embodiments.
[0071] 10, the X-ray imaging device of the fourth embodiment includes an optical flow detection unit 27 and an optical flow determination unit 28 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments. An alarm sound generation unit 29 is connected to the optical flow determination unit 28.
[0072] The camera image processing unit 20 of the X-ray imaging device of the fourth embodiment executes the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment to extract an image of the area around the skeleton of the subject 10, and then executes steps 113 to 116 of Fig. 11. This will be specifically described below.
[0073] (Steps 101-112) The camera image processing unit 20 executes any one of the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment. At this time, the subject skeleton determination unit 25 extracts an image of the area around the skeleton of the subject 10 in step 102, displays it, and stores it in the memory 25a. The image rotation unit 23 also stores the image 21a rotated in step 102 in the built-in memory.
[0074] (Step 113) The optical flow detection unit 27 calculates the amount of movement (optical flow) of corresponding pixels between the image 21a rotated in step 102 and the image 21a acquired and rotated at a timing a predetermined time before the image 21a was acquired (see Figure 12).
[0075] (Step 114) The optical flow determination unit 28 determines whether or not an optical flow is detected in the area surrounding the skeleton of the subject extracted in step 111. If an optical flow is detected in the area surrounding the skeleton of the subject, the process proceeds to step 115.
[0076] (Step 115) The optical flow determination unit 28 sounds an alarm if there are a predetermined number or more pixels whose optical flow (amount of movement) exceeds a threshold value among pixels in a region around a skeleton determined to be an object.
[0077] In this way, according to embodiment 5, not only is an image of subject 10 extracted and displayed, but it is also possible to detect whether subject 10 is moving, and if so, sound an alarm indicating the possibility of anesthesia wearoff.
[0078] Instead of an alarm sound, a warning may be issued by display or light.
[0079] <Embodiment 5> The X-ray imaging device according to the fifth embodiment will be described.
[0080] The X-ray imaging device of the fifth embodiment has a function of enlarging and displaying the face area of the image of the subject 10. By allowing a doctor to visually check the enlarged face of the subject 10 on the monitoring image display monitor 15, the doctor can easily monitor the condition of the subject.
[0081] The X-ray imaging device of the fifth embodiment includes a subject face area detection unit 31 and a subject face area enlargement unit 32 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments as shown in FIG.
[0082] The camera image processing unit 20 of the X-ray imaging device of the fifth embodiment executes the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment to extract an image of the area around the skeleton of the subject 10, and then executes steps 117 to 119 of Fig. 14. This will be specifically described below.
[0083] (Steps 101-112) The camera image processing unit 20 executes any one of the operations of the flow in FIG. 5 of embodiment 1, the flow in FIG. 6 of embodiment 2, and the flow in FIG. 8 of embodiment 3, extracts the area around the subject's skeleton, and displays it on the surveillance image display monitor 15.
[0084] (Step 117) 15 based on the positions of both shoulders and ears in the image of the subject 10 extracted in step 111. Specifically, the subject's face area detection unit 31 extracts, as the face area 141 of the subject 10, a rectangular area whose upper end is located at a position two times the distance from the lower end of the face area 141 to the ears in the Y direction and whose X coordinate of the left shoulder in the image of the subject 10 is the left end of the image of the face area 141, the X coordinate of the right shoulder is the right end of the image of the face area 141, and the Y coordinate of one of the shoulders is the lower end of the image of the face area 141, as the face area 141 of the subject 10.
[0085] (Step 118) The subject face area enlargement section 32 displays an enlargement button 142 in a corner of the screen of the monitoring image display monitor 15 (see FIG. 16(a)). When the doctor presses (or clicks, etc.) the enlargement button 142, the face area 141 is enlarged at a predetermined magnification and displayed on the monitoring image display monitor 15 (see FIG. 16(b)). The enlargement magnification may be a magnification calculated by the subject face area enlargement section 32 according to the size of the face area 141 so that the face area 141 is displayed over the entire screen of the monitoring image display monitor 15, or it may be a predetermined magnification.
[0086] According to the fifth embodiment, a doctor performing a procedure while viewing a fluoroscopic image on the fluoroscopic image display monitor 14 and an endoscopic image on the endoscopic image display monitor 16 can visually confirm the face of the subject 10 simply by moving his / her gaze to the monitoring image display monitor 15. Therefore, the doctor does not need to significantly lower his / her gaze in order to directly view the face of the subject 10, and can continue the procedure and can quickly notice any changes in the condition of the subject 10.
[0087] <Embodiment 6> An X-ray imaging apparatus according to a sixth embodiment will be described.
[0088] The X-ray imaging device of the sixth embodiment has a function of judging whether the subject 10 is about to fall off the top board 80, and sounding an alarm if the subject is about to fall. This makes it possible to warn the doctor that the subject 10 may fall off the top board 80 due to bodily movements caused by the effects of anesthesia, and prevents the subject from falling.
[0089] The X-ray imaging device of the sixth embodiment includes a shoulder / waist displacement amount calculation unit 33 and a fall possibility determination unit 34 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments as shown in FIG.
[0090] The camera image processing unit 20 of the X-ray imaging device of the sixth embodiment executes the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment to extract an image of the area around the skeleton of the subject 10, and then executes steps 120 to 122 of Fig. 18. This will be specifically described below.
[0091] (Steps 101-112) The camera image processing unit 20 executes any one of the operations of the flow in FIG. 5 of embodiment 1, the flow in FIG. 6 of embodiment 2, and the flow in FIG. 8 of embodiment 3, extracts the area around the subject's skeleton, and displays it on the surveillance image display monitor 15.
[0092] (Step 120) The shoulder / waist misalignment calculation unit 33 reads out from the memory 25a the coordinates of the shoulders and waist of the subject 10 determined by the subject's bone structure determination unit 25 and the coordinates of the shoulders and waist of the subject 10 determined for the previous frame, and calculates the amount of misalignment between frames. Note that the target for calculating the amount of misalignment is not limited to the previous frame, and may be a predetermined number of frames prior.
[0093] (Steps 120 and 121) The fall possibility determination unit 34 determines whether the amount of deviation calculated in step 120 is equal to or greater than a predetermined threshold, and if so, sounds an alarm.
[0094] In the sixth embodiment, the amount of deviation of both the shoulders and the waist is calculated, but it may be only one of them.
[0095] Also, instead of sounding an alarm, a warning may be issued by display or light.
[0096] <Embodiment 7> An X-ray imaging apparatus according to a seventh embodiment will be described.
[0097] The X-ray imaging device of embodiment 7 determines which part of the subject is irradiated with X-rays based on the skeleton of the subject detected by any of embodiments 1 to 3, and sets irradiation conditions and parameters for image generation to appropriate values according to the part of the subject.
[0098] The X-ray imaging device of embodiment 7 includes, in addition to the configuration of the X-ray imaging device of embodiments 1 to 3, an X-ray exposure area determination unit 35 and an X-ray exposure condition / image generation parameter calculation unit 36 within the camera image processing unit 20, as shown in FIG.
[0099] The camera image processing unit 20 of the X-ray imaging device of the seventh embodiment executes the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment to extract an image of the area around the skeleton of the subject 10, and then executes steps 123 to 126 of Fig. 20. This will be specifically described below.
[0100] (Steps 101-112) The camera image processing unit 20 executes any one of the operations of the flow in FIG. 5 of embodiment 1, the flow in FIG. 6 of embodiment 2, and the flow in FIG. 8 of embodiment 3, extracts the area around the subject's skeleton, and displays it on the surveillance image display monitor 15.
[0101] (Step 123) The X-ray irradiation area determination unit 35 receives the focus detector distance (SID) from the X-ray irradiation device 11, and also receives the amount of movement from the X-ray tilt mechanism 61, the X-ray up and down movement mechanism 60, and the tabletop movement mechanism 82, calculates the positional relationship between the tabletop 80 and the X-ray source in the X-ray irradiation device 11, and calculates the X-ray irradiation area in the image 21a.
[0102] (Step 124) Furthermore, the X-ray irradiation area determination unit 35 receives the coordinates of the skeleton of the subject 10 from the subject skeleton determination unit 25, determines which part of the subject is within the X-ray irradiation area, and finds the X-ray irradiated part 211 of the subject 10 (e.g., neck, chest, abdomen, waist, thighs, etc.) (see Figure 21).
[0103] (Step 125) The X-ray irradiation condition / image generation parameter calculation unit 36 refers to a built-in table and calculates the X-ray irradiation conditions (eg, tube current and tube voltage) and image generation parameters suitable for the X-ray irradiated area 211 obtained in step 123 .
[0104] (Step 126) The X-ray irradiation condition / image generation parameter calculation unit 36 sets X-ray irradiation conditions in the X-ray irradiator 11 and sets image generation parameters in the X-ray image generation unit 13 .
[0105] According to the seventh embodiment, it is possible to detect the part of the subject 10 in the irradiation field from the position of the skeleton, irradiate an appropriate amount of X-rays, and generate a fluoroscopic image with appropriate parameter values. Therefore, it is possible to prevent halation of the fluoroscopic image and improve the image quality.
[0106] <Embodiment 8> An X-ray imaging apparatus according to an eighth embodiment will be described.
[0107] The X-ray imaging apparatus of the eighth embodiment has a function of automatically moving the X-ray irradiator 11 to a site designated by a doctor, based on the skeleton of the subject detected by any one of the first to third embodiments.
[0108] The X-ray imaging device of the eighth embodiment includes an X-ray exposure region determining unit 35 and a difference calculating unit 37 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments as shown in FIG.
[0109] The camera image processing unit 20 of the X-ray imaging device of the eighth embodiment executes the operations of the flow of Fig. 5 of the first embodiment, the flow of Fig. 6 of the second embodiment, and the flow of Fig. 8 of the third embodiment to extract an image of the area around the skeleton of the subject 10, and then executes steps 127 to 131 of Fig. 23. This will be specifically described below.
[0110] (Steps 101-112) The camera image processing unit 20 executes any one of the operations of the flow in FIG. 5 of embodiment 1, the flow in FIG. 6 of embodiment 2, and the flow in FIG. 8 of embodiment 3, extracts the area around the subject's skeleton, and displays it on the surveillance image display monitor 15.
[0111] (Step 127) The X-ray irradiation area determination unit 35 receives the focus detector distance (SID) from the X-ray irradiation device 11, as well as the amount of movement from the X-ray tilt mechanism 61, the X-ray up and down movement mechanism 60, and the tabletop movement mechanism 82, calculates the positional relationship between the tabletop 80 and the X-ray source in the X-ray irradiation device 11, and calculates the X-ray irradiation area 240 in the image 21a (see Figure 24).
[0112] (Step 128) The difference calculation unit 37 receives an imaging designated region 242 of the subject 10 (for example, the neck, chest, abdomen, waist, thighs, knees, stomach, etc.) from the doctor.
[0113] (Step 128) Furthermore, the difference calculation section 37 receives the coordinates of the skeleton of the subject 10 from the subject skeleton determination section 25, and calculates the coordinates of the imaging designated region 242 from the coordinates of the subject's skeleton in the image 21a (see FIG. 24).
[0114] (Step 125) The difference calculation unit 37 calculates the difference in coordinates between the center 241 of the X-ray exposure region 240 calculated in step 127 and the center of the designated portion to be imaged 242 calculated in step 128 . Calculate.
[0115] (Step 126) The difference calculation unit 37 sets a movement amount corresponding to the difference calculated in step 125 for one or more of the X-ray tilt mechanism 61, the X-ray up / down movement mechanism 60, and the tabletop movement mechanism 82. As a result, the X-ray irradiator 11 moves relative to the tabletop 80, and the center 241 of the X-ray irradiation area 240 coincides with the center of the designated portion 242 to be imaged.
[0116] According to the eighth embodiment, the doctor does not need to move the X-ray irradiator 11 relative to the tabletop 80 while viewing the fluoroscopic image while irradiating X-rays and capturing the fluoroscopic image, so the X-ray exposure of the subject 10 and the doctor is reduced.
[0117] Moreover, it is possible to save the doctor the trouble of positioning the subject 10 and the X-ray irradiator 11 at a desired location. [Explanation of symbols]
[0118] 1 X-ray imaging device 10 Subject 11 X-ray irradiation device 12 Flat X-ray detector 13 X-ray image generation section 14 Fluoroscopic image display monitor 15 Surveillance image display monitor 16 Endoscope image display monitor 20 Camera image processing section 21 Surveillance Camera 21a Image 22 Image capture section 23 Image Rotation Unit 24 Skeleton detection section 25. Subject Skeleton Judgment Unit 25a Memory 26 Object region extraction section 27 Optical flow detection section 28 Optical flow determination unit 29 Alarm sound generating section 30 Operator 31 Subject face area detection unit 32 Subject's face area enlargement section 33 Shoulder / waist displacement calculation unit 34 Fall possibility determination section 35 X-ray irradiation area determination section 36 Image generation parameter calculation unit 37 Difference calculation part 40 Candidates 45 Line connecting both shoulders 50 X-ray support 60 X-ray vertical movement mechanism 61 X-ray tilt mechanism 62 Pressing tool 70 Stand 80 Tabletop 81 Top plate support 82 Top plate movement mechanism 135 Skeleton Centerline 136 Image centerline 141 Face Area 142 Enlarge button 211 X-ray irradiation area 240 X-ray irradiation area 241 center 242 Imaging designated area L distance W Shoulder width Oi center Os midpoint
Claims
1. the apparatus includes a tabletop on which an object is placed, an X-ray irradiator that irradiates the object with X-rays, an X-ray detector that detects the X-rays that are irradiated from the X-ray irradiator and that have passed through the object, and an image processor that receives an image from a camera that captures an image of the object and processes the image; The image processing unit detects the skeletons of multiple human images contained in the image, extracts an image of the subject placed on the tabletop from among the multiple human images based on the detected skeletons, and displays the extracted image of the subject on a connected display device, characterized in that the image processing unit detects the skeletons of multiple human images contained in the image, extracts an image of the subject placed on the tabletop from among the multiple human images based on the detected skeletons, and displays the extracted image of the subject on a connected display device.
2. 2. The X-ray imaging apparatus according to claim 1, wherein the image processing unit determines that the image of the person having the smallest angle between a center line of the skeleton of the person image and a center line of the image is the image of the subject.
3. 2. The X-ray imaging apparatus according to claim 1, wherein the image processor determines that the image of the person having the greatest distance between both shoulders of the skeleton of the person is the image of the subject.
4. 2. The X-ray imaging device according to claim 1, wherein the image processing unit determines, as the image of the subject, the image of the person for which at least one of a distance L between the midpoint of a line connecting both shoulders of the skeleton of the person image and the center of the image and an angle δ between the line connecting both shoulders and a horizontal line of the image is smallest.
5. 2. The X-ray imaging device according to claim 1, wherein the image processing unit calculates the amount of movement of corresponding pixels in a region surrounding the image of the subject between the image and an image acquired by the camera at a timing a predetermined time before the timing of acquiring the image, and issues an alarm if there are a predetermined number or more pixels whose amount of movement exceeds a threshold.
6. 2. The X-ray imaging apparatus according to claim 1, wherein said image processing section further extracts a face area of said subject from said image, enlarges it and displays it on a display device.
7. 2. The X-ray imaging device according to claim 1, wherein the image processing unit calculates an amount of deviation for at least one of the shoulder and waist positions between the image of the subject in the image and the image of the subject in an image acquired by the camera at a timing a predetermined time before the timing of acquisition of the image, and issues an alarm if the amount of deviation is equal to or greater than a predetermined threshold value.
8. 2. The X-ray imaging device according to claim 1, wherein the image processing unit determines a part of the subject being irradiated with X-rays from the X-ray irradiator based on a skeleton of the subject in the image, and sets X-ray irradiation conditions for the X-ray irradiator in accordance with the part.
9. 2. The X-ray imaging apparatus according to claim 1, further comprising a mechanism for moving the X-ray irradiator relative to the top plate, the image processing unit calculates coordinates in the image of a designated imaging portion designated by the subject for imaging based on the skeleton of the subject in the image, calculates the coordinates of a center of an X-ray irradiation area in the image, calculates a difference between the coordinates of the designated imaging portion and the X-ray irradiation area, sets a movement amount corresponding to the difference in the mechanism unit, sets the movement amount in the mechanism unit to move the X-ray irradiator relatively to the tabletop, and moves the center of the X-ray irradiation area to the designated imaging portion.
10. A method for processing an image captured by a camera of an X-ray imaging device, comprising the steps of: Detecting skeletons of a plurality of human figures included in an image captured by the camera; An image of a subject placed on a tabletop is extracted from the plurality of human images based on the detected skeleton, and the extracted image of the subject is displayed on a connected display device.
13. An image processing method for an X-ray imaging device equipped with a camera, comprising:
Citation Information
Patent Citations
Radiographic system and control method thereof, and program
JP2021186199A
Skeleton detection system and work management device
JP2021196783A
