X-ray imaging apparatus and image processing method
By using the camera image processing unit in the X-ray camera device to detect and extract bone feature points in the surveillance camera, the problem that doctors find it difficult to quickly confirm the patient's condition is solved, and the doctors can visually recognize the patient's status without looking down, improving the operation Safety and efficiency of the process.
Patent Information
- Application Number
- CN202410423835.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-26
- Filing Date
- 2024-04-09
- Publication Date
- 2024-12-03
AI Technical Summary
In interventional radiation therapy, it is difficult for doctors to quickly and accurately confirm the patient's condition while observing the fluoroscopic image, especially when there are characters other than the patient in the surveillance camera image, which makes it difficult to grasp the patient's status in a timely manner.
By using the camera image processing unit in the X-ray camera device, the image processing unit is used to receive images from the surveillance camera, detect and extract bone feature points in a plurality of person images, and identify and display images of the subject mounted on the upper plate, so as to realize the inspection The visual identification of the body prevents the doctor from moving his eyes from the monitor to the patient.
Doctors can identify the patient's status through surveillance camera images without downward vision, which improves the visibility and timeliness of the patient's status and ensures safety and efficiency during the operation.
Smart Images

Figure CN119055265A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an X-ray imaging device for acquiring an image by irradiating an object with X-rays. Background Art
[0002] There is known an IVR (Interventional Radiology) in which an endoscope or a catheter is inserted into the patient's body while irradiating the patient with X-rays from an X-ray camera to obtain a fluoroscopic image, and various surgeries are performed on the patient. The doctor performs the surgery on the patient while observing a monitor showing the fluoroscopic image and a monitor showing the endoscopic image.
[0003] When confirming the patient's condition during surgery while observing the fluoroscopic image, the doctor moves his gaze to the patient's face, feet, or hands for visual confirmation, or moves his gaze to a monitor installed on the ceiling or upper wall of the room that displays the image of a surveillance camera and confirms the patient through the surveillance camera image.
[0004] On the other hand, Patent Document 1 discloses a technique for detecting body motion of a subject by performing image processing on a fluoroscopic image.
[0005] Patent Document 2 discloses a technique of capturing images of a plurality of operators performing an operation using a camera and processing the images to perform skeleton detection, thereby identifying an operator performing a specific operation from among the plurality of operators.
[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-186199
[0007] Patent Document 2: Japanese Patent Application Publication No. 2021-196783
[0008] In IVR, the patient is usually placed next to the doctor, and the display of fluoroscopic images and endoscopic images is placed in front of the doctor. Therefore, in order to visually confirm the patient's condition, the doctor needs to move his eyes from the display to the patient.
[0009] On the other hand, when the doctor checks the patient's condition by viewing the monitor of the surveillance camera image in the room, it is difficult to immediately understand the patient's condition because the doctor, technicians, and nurses are reflected in the surveillance camera image in addition to the patient.
[0010] Furthermore, it is conceivable to limit the range in which the monitoring camera image is displayed to prevent people other than the patient from being displayed on the monitor. However, if the position or orientation of the upper board carrying the patient is changed as the operation progresses, the monitoring camera may not be able to capture the subject. Summary of the invention
[0011] An object of the present invention is to extract and display an image of a subject on an upper plate on which a fluoroscopic image is being captured, from an image captured by a monitoring camera.
[0012] In order to achieve the above-mentioned object, according to the present invention, there is provided an X-ray imaging device, which comprises: an upper plate for carrying a subject; an X-ray irradiation device for irradiating the subject with X-rays; an X-ray detector for detecting X-rays irradiated from the X-ray irradiation device and transmitted through the subject; and a camera image processing unit for receiving an image from a camera photographing the subject and processing the image. The image processing unit detects the skeletons of multiple human figures included in the image, extracts the image of the subject carried on the upper plate from the multiple human figures based on the detected skeletons, and displays the image on a connected display device.
[0013] Effects of the Invention
[0014] According to the present invention, since the image of the patient on the upper plate where the fluoroscopic image is taken can be extracted from the image taken by the monitoring camera and displayed on the display, the doctor can visually recognize the patient's condition through the display of the monitoring camera image without looking down at the patient. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 (a) is a front view of the X-ray imaging device 1 according to the first embodiment, Figure 1 (b) Figure 1 (c) is a diagram showing an image displayed on a monitoring image display.
[0016] Figure 2 This is a functional block diagram of the X-ray imaging device according to the first embodiment.
[0017] Figure 3 The diagram shows an image captured by the monitoring camera of the X-ray imaging device according to the first embodiment and bones detected from the image.
[0018] Figure 4 This is an image diagram showing an area surrounding the bones of the subject 10 extracted from an image captured by the monitoring camera of the X-ray imaging device according to the first embodiment.
[0019] Figure 5 This is a flowchart showing the operation of the X-ray imaging device according to the first embodiment.
[0020] Figure 6 This is a flowchart showing the operation of the X-ray imaging device according to the second embodiment.
[0021] Figure 7 This is a diagram showing an image captured by a monitoring camera of an X-ray imaging device according to a second embodiment and bones detected from the image.
[0022] Figure 8 This is a flowchart showing the operation of the X-ray imaging device according to the third embodiment.
[0023] Fig. 9 This is a diagram showing an image captured by a monitoring camera of an X-ray imaging device according to a third embodiment and bones detected from the image.
[0024] Fig.10 This is a functional block diagram of an X-ray imaging device according to a fourth embodiment.
[0025] Fig.11 This is a flowchart showing the operation of the X-ray imaging device according to the fourth embodiment.
[0026] Fig.12 It is a diagram showing an image captured by a monitoring camera of an X-ray imaging device according to a fourth embodiment and optical flows detected from the image.
[0027] Fig.13 This is a functional block diagram of an X-ray imaging device according to the fifth embodiment.
[0028] Fig.14 This is a flowchart showing the operation of the X-ray imaging device according to the fifth embodiment.
[0029] Fig.15 It is a diagram showing an image captured by a monitoring camera of an X-ray imaging device according to the fifth embodiment and optical flows detected from the image.
[0030] Fig.16 (a) is a diagram showing a display screen of an image captured by a monitoring camera of an X-ray imaging device according to Embodiment 5, Fig.16 (b) is a diagram of a display screen showing an enlarged view of the face region of an image of a subject.
[0031] Fig.17 This is a functional block diagram of an X-ray imaging device according to a sixth embodiment.
[0032] Fig.18 This is a flowchart showing the operation of the X-ray imaging device according to the sixth embodiment.
[0033] Fig.19 This is a functional block diagram of an X-ray imaging device according to the seventh embodiment.
[0034] Fig. 20 This is a flowchart showing the operation of the X-ray imaging device according to the seventh embodiment.
[0035] Fig.21 The diagram shows an image captured by a monitoring camera of an X-ray imaging device according to a seventh embodiment, and bones and X-ray irradiated parts detected from the image.
[0036] Fig. 22 This is a functional block diagram of an X-ray imaging device according to the eighth embodiment.
[0037] Fig.23 This is a flowchart showing the operation of the X-ray imaging device according to the eighth embodiment.
[0038] Fig.24 It is a diagram showing an image captured by a monitoring camera of the X-ray imaging device of embodiment 8, and bones and X-ray irradiated parts detected from the image.
[0039] Explanation of symbols
[0040] 1-X-ray imaging device, 10-subject, 11-X-ray irradiation device, 12-planar X-ray detector, 13-X-ray image generating unit, 14-fluoroscopic image display, 15-monitoring image display, 16-endoscopic image display, 20-camera image processing unit, 21-monitoring camera, 21a-image, 22-image reading unit, 23-image rotation unit, 24-bone detection unit, 25-subject bone determination unit, 25a-memory, 26-subject region extraction unit, 27-optical flow detection unit, 28-optical flow determination unit, 29-alarm sound generating unit, 30-operator, 31-subject face region detection unit, 32-subject face region magnification unit, 33-shoulder / waist offset calculation unit, 34 -Fall possibility determination unit, 35-X-ray irradiation area determination unit, 36-X-ray irradiation condition / image generation parameter calculation unit, 37-difference calculation unit, 40-candidate subject, 45-line connecting the two shoulders, 50-X-ray supporting unit, 60-X-ray up and down moving mechanism unit, 61-X-ray tilting mechanism unit, 62-pressing tool, 70-support, 80-upper plate, 81-upper plate supporting unit, 82-upper plate moving mechanism unit, 135-bone center line, 136-image center line, 141-face area, 142-zoom in button, 211-X-ray irradiation part, 240-X-ray irradiation area, 241-center, 242-designated imaging part, L-distance, W-shoulder width, Oi-center, Os-midpoint. DETAILED DESCRIPTION
[0041] Hereinafter, an X-ray irradiation device according to an embodiment of the present invention will be described using the drawings.
[0042] <Implementation Method 1>
[0043] use Figure 1 to Figure 3 , an X-ray imaging device according to embodiment 1 is described. Figure 1 (a) is a front view of the X-ray imaging device 1, Figure 1 (b) Figure 1(c) is an image displayed on the monitoring image display. Figure 2 2 is a functional block diagram of the X-ray imaging device 1 . Figure 3 It is a graph showing an image captured by a surveillance camera and bones detected from the image.
[0044] like Figure 1 As shown in (a), the X-ray imaging device 1 includes an upper plate 80 on which the subject 10 is mounted and an X-ray irradiation device 11 for irradiating the subject 10 with X-rays. The X-ray irradiation device 11 is supported on a support 70 via an X-ray support 50. The support 70 includes: an X-ray vertical movement mechanism 60 for moving the X-ray support 50 up and down; a long-side direction movement mechanism (not shown) for moving the X-ray support 50 along the long-side direction of the upper plate 80; a short-side direction movement mechanism (not shown) for moving the X-ray support 50 along the short-side direction of the upper plate 80; and an X-ray tilting mechanism 61 for tilting the central axis of the X-ray support 50 around the short-side direction of the upper plate. In addition, an X-ray rotation mechanism (not shown) for rotating the X-ray irradiation device 11 around the long-side direction of the upper plate is provided in the X-ray support 50. A pressing tool 62 for pressing the subject 10 is provided in the X-ray support 50.
[0045] On the other hand, the upper plate 80 is supported by the bracket 70 via the upper plate support portion 81 .
[0046] The planar X-ray detector 12 is disposed inside the upper plate support portion 81. The planar X-ray detector 12 is a planar detector in which X-ray detection elements are two-dimensionally arranged. The X-ray detector 12 detects X-rays irradiated from the X-ray irradiation device 11 and transmitted through the subject 10.
[0047] A monitoring camera 21 for photographing the subject 10 is provided in the X-ray support unit 50 or the X-ray irradiation device 11. The monitoring camera 21 is provided at a position where the X-rays irradiated from the X-ray irradiation device 11 are not blocked. Figure 1 As shown in (b) of FIG. 8 , the viewing angle of the monitoring camera 21 is set so as to be able to capture the entire body of the subject 10 on the upper plate 80 . In addition, the monitoring camera 21 may be fixed to the cover (housing) of the X-ray irradiation device 11 .
[0048] like Figure 2 As shown, an X-ray image generator 13 is connected to the planar X-ray detector 12. The X-ray image generator 13 receives the signal of the X-rays detected and output by each X-ray detection element of the X-ray detector 12, and generates an X-ray image (here, a fluoroscopic image) at a predetermined frame rate. A fluoroscopic image display 14 is connected to the X-ray image generator 13. The fluoroscopic image is displayed on the fluoroscopic image display 14.
[0049] On the other hand, a camera image processing unit 20 is connected to the monitoring camera 21. The camera image processing unit 20 receives an image from the monitoring camera 21 and processes the image. Specifically, the camera image processing unit 20 receives an image from the monitoring camera 21, detects the skeletons of a plurality of human figures included in the received image, and extracts the image of the subject 10 mounted on the upper plate 80 from the plurality of human figures based on the detected skeletons. The camera image processing unit 20 displays the extracted image of the subject 10 on the connected monitoring image display 15.
[0050] The camera image processing unit 20 includes an image reading unit 22 , an image rotating unit 23 , a subject skeleton determining unit 25 , and a subject region extracting unit 26 .
[0051] The image reading unit 22 reads the image 21a outputted from the monitoring camera 21 at a predetermined time interval. The predetermined time interval can be set to a level similar to the frame rate of the fluoroscopic image generated by the X-ray image generating unit 13 .
[0052] The image 21 a is an image captured so that the body axis of the subject on the upper plate 80 is located on the horizontal axis of the image.
[0053] The image rotation unit 23 rotates the image 21a by 90 degrees in a predetermined direction, and Figure 3 As shown, the head position of the image of the subject 10 on the upper plate 80 is located above the image 21a. Since the direction in which the head of the subject 10 is placed on the upper plate 80 is predetermined, the direction of rotation can be predetermined by the predetermined head position on the upper plate 80 and the direction of the monitoring camera 21.
[0054] The skeleton detection unit 24 is as follows Figure 3 As shown, the feature points of all bones in the image 21a (two ears, nose, two eyes, two shoulders, two elbows, two wrists, two waists, two knees, two ankles, etc.) are extracted. The skeleton detection unit 24 detects all bones by detecting the two-dimensional coordinates of these. The skeleton detection unit 24 can be implemented by a well-known posture detection algorithm. For example, the posture estimation algorithm Open Pose (registered trademark) included in Open VINO (registered trademark) provided by IntelCorp can be used.
[0055] The subject skeleton determination unit 25 sets a person whose ears, shoulders, and waists are detected among the characteristic points of the skeleton detected by the skeleton detection unit 24 as a candidate subject 40. A person whose ears, shoulders, and waists are detected is identified as a candidate subject 40 even if the person is standing (for example, the operator 30).
[0056] The subject skeleton determination unit 25 calculates the center line of the skeleton for each candidate subject. Specifically, a line connecting the middle point of the two shoulders and the middle point of the two waists is calculated, and this calculated line is used as the skeleton center line 135. The subject skeleton determination unit 25 calculates the angle θ formed by the calculated skeleton center line 135 and the longitudinal center line 136 of the image 21a rotated by the image rotation unit 23. The subject skeleton determination unit 25 compares the angle θ of multiple candidate subjects 40, and determines the subject with the smallest angle θ as the candidate subject 40. That is, the subject whose skeleton center line 135 is closest to the center line 136 of the image 21a is determined to be the subject 10.
[0057] The subject region extraction unit 26 extracts the image 21a as follows: Figure 4 The skeleton determined to be the subject 10 and its surrounding area are extracted. Specifically, based on the coordinates of the feature points of the skeleton of the subject 10, the coordinates of the left end, right end, top end and bottom end of the rectangular area including the subject 10 are determined, and the rectangular area is extracted. For example, when the horizontal direction of the rotated image 21a is taken as the x-axis and the vertical direction is taken as the y-axis, among the feature points of the detected skeleton (two ears, nose, two eyes, two shoulders, two elbows, two wrists, two waists, two knees, two ankles), the coordinates of the feature point located on the leftmost side in the x-axis direction to the left by a specified distance (for example: 30 pixels) are taken as the left end of the area, the coordinates of the feature point located on the rightmost side in the x-axis direction to the right by a specified distance (for example: 30 pixels) are taken as the right end of the area, and the coordinates of the feature point located on the bottom side in the y-axis direction to the bottom by a specified distance (for example: 30 pixels) are taken as the bottom end of the area. Furthermore, when the y coordinate of the ear located at the lower side in the y-axis direction of the two ears is set as Ye, and the y coordinate of the shoulder located at the upper side in the y-axis direction of the two shoulders is set as Ys, the position of Ye-(Ys-Ye) is set as the upper end of the region. The subject region extraction unit 26 extracts the rectangular region, rotates the image of the extracted region of the subject 10 by 90 degrees in a predetermined direction, and then displays it on the monitoring image display 15. The direction in which the subject region extraction unit 26 rotates the image of the region of the subject 10 by 90 degrees is the opposite direction to the direction in which the image rotation unit 23 rotates the image 21a.
[0058] Below, use Figure 5 The operation of each unit of the camera image processing unit 20 is described below.
[0059] In addition, the functions of each unit (22 to 26) of the camera image processing unit 20 can be realized by software. In this case, the camera image processing unit 20 is composed of a computer or the like having a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) and a memory, and the CPU reads and executes the program stored in the memory to realize these functions. In addition, part or all of the camera image processing unit 20 can also be composed of hardware. For example, a custom IC such as an ASIC (Application Specific Integrated Circuit) or a programmable IC such as an FPGA (Field-Programmable Gate Array) can be used to perform circuit design to realize the functions of each unit.
[0060] (Step S101)
[0061] The image reading unit 22 reads the image 21a (see Figure 1 (b)). The time point when the image reading unit 22 starts reading the image may be the time point when the start instruction of the operator is received, or the time point when the endoscope image display 16 starts displaying the endoscope image.
[0062] (Step S102)
[0063] The image rotating unit 23 rotates the image 21a by 90 degrees in a predetermined direction. As a result, the end of the head of the predetermined subject 10 on the upper plate 80 is placed above the image 21a.
[0064] (Step S103)
[0065] The skeleton detection unit 24 extracts the characteristic points of all the skeletons (two ears, nose, two eyes, two shoulders, two elbows, two wrists, two waists, two knees, two ankles, etc.) reflected in the image 21a, and detects the two-dimensional coordinates of these (reference Figure 3 ).
[0066] (Steps S104, S105)
[0067] The subject skeleton determination unit 25 determines whether or not there is a person whose ears, shoulders, and waist are detected among the skeletons detected by the skeleton detection unit 24 , and identifies the person as the candidate subject 40 .
[0068] (Step S106)
[0069] The subject skeleton determination unit 25 calculates a skeleton center line 135 (reference line 135 ) connecting the middle points of the two shoulders and the middle points of the two waists for the candidate subject 40 identified in step S105. Figure 3 ).
[0070] (Step S107)
[0071] The subject skeleton determination unit 25 calculates an angle θ (reference angle θ) formed between the calculated skeleton center line 135 and the longitudinal center line 136 of the image 21a rotated in step S102. Figure 3 ).
[0072] (Steps S108, S109)
[0073] The subject skeleton determination unit 25 compares the angle θ of the candidate subject 40 calculated in step S107 with the angle θ of the candidate subject stored in the built-in memory 25a. When the angle θ calculated in step S107 is smaller than the angle θ in the memory 25a, the angle θ in the memory 25a is overwritten with the angle θ calculated in step S107 to save (update), and the candidate subject in the memory 25a is overwritten with the coordinates of the feature points of the candidate subject 40 for which the angle θ is calculated in step S107 to update. In addition, since the initial candidate subject 40 is in a state where the angle θ is not stored in the memory 25a, the angle θ calculated in step S107 and the candidate subject 40 are stored in the memory 25a without comparison.
[0074] (Steps S110, S111)
[0075] The subject skeleton determination unit 25 determines whether there are any bones among the bones detected in step S103 that have not been determined in step S104. If there are any bones that have not been determined in step S104, the process returns to step S104 to determine whether the ears, shoulders, and waists have been detected, and steps S105 to S109 are repeated. On the other hand, in step S110, if there are no bones that have not been determined in step S104, the process proceeds to step S111 and extracts the image of the surrounding area of the skeleton of the subject 10 stored in the current memory 25a (reference Figure 4 ).
[0076] (Step S112)
[0077] The subject skeleton determination unit 25 rotates the image of the region of the subject 10 extracted in step S111 by 90 degrees in the opposite direction to that in step S102 and displays the image on the monitoring image display 15 (see Figure 1 (c)).
[0078] In the X-ray imaging device of the first embodiment, in IVR (Interventional Radiology), the monitoring image display 15 is arranged side by side with the fluoroscopic image display 14 and the endoscopic image display 16, so that the doctor can visually recognize the condition of the subject 10 by only slightly moving the line of sight to the monitoring image display 15 next to the monitor image display 15 while observing the fluoroscopic image and the endoscopic image on the displays 14 and 16. Thus, the doctor can visually recognize the subject 10 without moving the line of sight from the displays 14 and 16 to the subject 10 on the upper plate 80.
[0079] The image displayed on the monitoring image display 15 is an image of the skeleton of the subject 10 and its surrounding area, so even if the doctor, technician, or nurse is reflected in addition to the subject 10 in the monitoring camera image, only the subject 10 can be confirmed on the monitoring image display 15. Therefore, the doctor can immediately understand the patient's condition through the monitoring image display 15.
[0080] Furthermore, since the monitoring camera 21 is provided on the X-ray support 50 or the X-ray irradiation device 11, even if the operator moves the position and direction of the X-ray irradiation device 11 as the operation proceeds, the monitoring camera 21 will still follow the X-ray irradiation device 11. Therefore, the monitoring camera 21 can always capture the image of the subject 10.
[0081] <Implementation Method 2>
[0082] The X-ray imaging device according to the second embodiment will be described.
[0083] The structure of the X-ray imaging device of embodiment 1 is to determine the bone with the smallest angle between the bone center line 135 and the image center line 136 in image 21a as the subject 10, but the X-ray imaging device of embodiment 2 determines the image with the largest bone (for example, shoulder width) on image 21a as the subject 10.
[0084] Therefore, the amount of calculation of the camera image processing unit 20 of the image processing device of the second embodiment is less than that of the first embodiment, and the processing speed is high.
[0085] The configuration of the X-ray imaging device of the second embodiment is the same as that of the first embodiment, but the operation of the subject skeleton determination unit 25 of the camera image processing unit 20 is different from that of the first embodiment. Figure 6 The process of the embodiment 1 is mainly used to explain the operation of the camera image processing unit 20.
[0086] (Steps S101 to S103)
[0087] The image reading unit 22 reads an image from the monitoring camera 21, and the image rotating unit 23 rotates the image 21a by 90 degrees. The skeleton detecting unit 24 extracts feature points of all skeletons reflected in the image 21a.
[0088] (Steps S204, S205)
[0089] The subject skeleton determination unit 25 determines whether there is a person with ears and shoulders among the skeletons detected by the skeleton detection unit 24, and identifies the person as a candidate subject 40. In the second embodiment, unlike the first embodiment, detection of waists is not a necessary condition for determination.
[0090] (Step S206)
[0091] The subject skeleton determination unit 25 calculates the distance between the two shoulders (shoulder width) W (reference Figure 7 ).
[0092] (Steps S207, S208)
[0093] The subject skeleton determination unit 25 compares the shoulder width W of the candidate subject 40 calculated in step S206 with the shoulder width W of the candidate subject stored in the built-in memory 25a. When the shoulder width W calculated in step S206 is larger than the shoulder width W in the memory 25a, the shoulder width W in the memory 25a is overwritten with the shoulder width W calculated in step S206 to be stored (updated), and the candidate subject in the memory 25a is overwritten with the coordinates of the feature points of the candidate subject 40 whose shoulder width W is calculated in step S206 to be stored and updated. In addition, since the initial candidate subject 40 is in a state where the shoulder width W is not stored in the memory 25a, the shoulder width W calculated in step S206 and the candidate subject 40 are stored in the memory 25a without comparison.
[0094] (Steps S110 to S112)
[0095] The subject skeleton determination unit 25 determines whether there are any bones among the bones detected in step S103 that have not been determined in step S204. If there are any bones that have not been determined, the process returns to step S204. On the other hand, in step S110, if there are no bones that have not been determined, the process proceeds to step S111, and an image of the surrounding area of the skeleton of the subject 10 stored in the current memory 25a is extracted and rotated 90 degrees in a specified direction to be displayed on the monitoring image display 15.
[0096] The X-ray imaging apparatus of the second embodiment determines whether or not the candidate subject 40 is the subject 10 based on the size of the body shape on the image 21 a , and thus can determine the subject with less calculation amount than the first embodiment.
[0097] Furthermore, the X-ray imaging device of Embodiment 2 does not use the waist bones for judging the subject 10, and therefore can judge the subject 10 even when the waist of the subject 10 is covered with cloth or the like and is not photographed by the monitoring camera 21 during IVR (Interventional Radiology).
[0098] The other structures and effects are the same as those in Implementation 1, and thus the description thereof will be omitted.
[0099] <Implementation Method 3>
[0100] An X-ray imaging device according to the third embodiment will be described.
[0101] The X-ray imaging apparatus of the first embodiment determines the subject 10 by the angle formed by the skeleton center line 135 and the image center line 136 in the image 21a. Therefore, when the subject 10 and the operator or doctor face the same direction, it is difficult to detect the subject 10.
[0102] On the other hand, in the second embodiment, the subject is determined by the size of the bones in the image 21a, so it is difficult to detect a small subject 10 such as a child.
[0103] Therefore, the X-ray imaging apparatus according to the third embodiment determines the subject using the distance from the center of the image 21a to the skeleton and the angle of the line connecting the two shoulders.
[0104] The structure of the X-ray imaging device of the third embodiment is the same as that of the first embodiment, but the operation of the subject skeleton determination unit 25 of the camera image processing unit 20 is different from that of the first embodiment. Figure 8 The process of the embodiment 1 is mainly used to explain the operation of the camera image processing unit 20.
[0105] (Steps S101 to S103)
[0106] The image reading unit 22 reads an image from the monitoring camera 21, and the image rotating unit 23 rotates the image 21a by 90 degrees. The skeleton detecting unit 24 extracts feature points of all skeletons reflected in the image 21a.
[0107] (Steps S304, S305)
[0108] The subject skeleton determination unit 25 determines whether or not there is a person whose ears and shoulders are detected among the skeletons detected by the skeleton detection unit 24 , and identifies the person as a candidate subject 40 .
[0109] (Step S306)
[0110] The subject skeleton determination unit 25 calculates the distance L (reference distance L) between the middle point Os of the line connecting the two shoulders and the center Oi of the image 21a rotated in step S102 for the candidate subject 40 identified in step S305. Fig. 9 ).
[0111] (Step S307)
[0112] The subject skeleton determination unit 25 calculates the angle δ (reference δ) between the line 45 connecting the two shoulders and the horizontal direction of the image for the candidate subject 40 identified in step S305. Fig. 9 ).
[0113] (Steps S308, S309)
[0114] The subject skeleton determination unit 25 determines whether the distance L calculated in step S306 and the angle δ calculated in step S308 are respectively smaller than the distance L and the angle δ of the candidate subject stored in the built-in memory 25a. If they are smaller, the distance L and the angle δ in the memory 25a are overwritten with the distance L and the angle δ calculated in steps S306 and S307 and stored (updated), and the candidate subject in the memory 25a is overwritten with the coordinates of the feature points of the candidate subject 40 for which the distance L and the angle δ are calculated in steps S306 and S307 to store and update.
[0115] Here, a human image whose distance L and angle δ are both smaller than those of other candidate subjects is determined to be the image of the subject 10 . However, a human image whose distance L and angle δ are smaller than those of other candidate subjects may be determined to be the image of the subject 10 .
[0116] (Steps S110 to S112)
[0117] The subject skeleton determination unit 25 determines whether there are any bones among the bones detected in step S103 that have not been determined in step S204. If there are any bones that have not been determined, the process returns to step S204. On the other hand, in step S110, if there are no bones that have not been determined, the process proceeds to step S111, and an image of the surrounding area of the skeleton of the subject 10 stored in the current memory 25a is extracted and rotated 90 degrees in a specified direction to be displayed on the monitoring image display 15.
[0118] Thus, even when the operator or doctor faces the same direction as the subject 10 or the subject 10 is small, the subject 10 can be detected with high accuracy to extract an image and display it on the monitoring image display 15 .
[0119] <Implementation Method 4>
[0120] An X-ray imaging device according to a fourth embodiment will be described.
[0121] The X-ray imaging apparatus according to the fourth embodiment further processes the image of the subject 10 extracted in any one of the first to third embodiments to detect release of anesthesia in the subject 10 .
[0122] like Fig.10 As shown, the X-ray imaging device of the fourth embodiment includes, in addition to the configuration of the X-ray imaging device of the first to third embodiments, an optical flow detection unit 27 and an optical flow determination unit 28 in the camera image processing unit 20. The optical flow determination unit 28 is connected to an alarm sound generation unit 29.
[0123] The camera image processing unit 20 of the X-ray imaging device of the fourth embodiment executes the Figure 5 The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 After extracting the image of the surrounding area of the skeleton of the subject 10, execute Fig.11 The steps S113 to S116 are described in detail below.
[0124] (Steps S101 to S112)
[0125] The camera image processing unit 20 executes the first embodiment Figure 5 The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 At this time, the subject skeleton determination unit 25 extracts and displays the image of the surrounding area of the skeleton of the subject 10 in step S102 and stores it in the memory 25a. In addition, the image rotation unit 23 stores the image 21a rotated in step S102 in the built-in memory.
[0126] (Step S113)
[0127] The optical flow detection unit 27 calculates, for each pixel, the movement amount (optical flow) of the corresponding pixels between the image 21a rotated in step S102 and the image 21a acquired and rotated at a predetermined time point earlier than the acquisition time point (refer to Fig.12 ).
[0128] (Step S114)
[0129] The optical flow determination unit 28 determines whether an optical flow is detected in the region around the skeleton of the subject extracted in step S111. When an optical flow is detected in the region around the skeleton of the subject, the process proceeds to step S115.
[0130] (Step S115)
[0131] The optical flow determination unit 28 sounds an alarm when the number of pixels whose optical flow (movement amount) exceeds a threshold value for pixels determined to be in the bone surrounding region of the subject is equal to or greater than a predetermined number.
[0132] As described above, according to the fifth embodiment, not only is the image of the subject 10 extracted and displayed, but also whether the subject 10 is moving is detected. If the subject is moving, it is determined that there is a possibility that anesthesia is released, and an alarm is issued.
[0133] Alternatively, a configuration may be adopted in which an alarm is issued by display or light instead of the alarm sound.
[0134] <Implementation method 5>
[0135] The X-ray imaging device according to the fifth embodiment will be described.
[0136] The X-ray imaging apparatus of the fifth embodiment has a function of magnifying and displaying the face region of the image of the subject 10. The doctor can easily monitor the condition of the subject by visually recognizing the magnified face of the subject 10 on the monitoring image display 15.
[0137] like Fig.13 As shown, the X-ray imaging apparatus of the fifth embodiment includes a subject face region detecting unit 31 and a subject face region magnifying unit 32 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging apparatus of the first to third embodiments.
[0138] The camera image processing unit 20 of the X-ray imaging device of the fifth embodiment executes the Figure 5 Process and implementation method 2 Figure 6 The process and implementation method of 3 Figure 8 After extracting the image of the surrounding area of the skeleton of the subject 10, execute Fig.14 The steps S117 to S119 are described in detail below.
[0139] (Steps S101 to S112)
[0140] The camera image processing unit 20 executes the first embodiment Figure 5 Process and implementation method 2 Figure 6 The process and implementation method of 3 Figure 8 By performing any of the actions in the process, the surrounding area of the subject's bones is extracted and displayed on the monitoring image display 15.
[0141] (Step S117)
[0142] The subject face region detecting unit 31 detects the positions of the shoulders and ears of the subject 10 image extracted in step S111, such as Fig.15As shown, the face region 141 of the subject 10 is detected. Specifically, the X coordinate of the left shoulder of the image of the subject 10 is set as the left end of the image of the face region 141, the X coordinate of the right shoulder is set as the right end of the image of the face region 141, the Y coordinate of either shoulder is set as the lower end of the image of the face region 141, and a quadrilateral region whose upper end is a position in the Y direction that is twice the distance from the lower end of the face region 141 to the ear is extracted as the face region 141 of the subject 10.
[0143] (Step S118)
[0144] The subject face region magnifying unit 32 displays a magnification button 142 (see FIG. Fig.16 If the doctor presses (or clicks, etc.) the zoom button 142, the face area 141 is magnified at a predetermined magnification and displayed on the monitoring image display 15 (refer to Fig.16 (b)). The magnification may be a magnification calculated by the subject face region magnifying unit 32 so that the face region 141 is displayed on the entire monitoring image display 15 according to the size of the face region 141, or may be a predetermined magnification.
[0145] According to Embodiment 5, a doctor who performs surgery while observing the fluoroscopic image on the fluoroscopic image display 14 or the endoscopic image on the endoscopic image display 16 can visually recognize the face of the subject 10 simply by shifting his line of sight to the monitoring image display 15. Therefore, the doctor can detect changes in the state of the subject 10 as early as possible while continuing the surgery without significantly lowering his line of sight in order to directly see the face of the subject 10.
[0146] <Implementation Method 6>
[0147] An X-ray imaging device according to the sixth embodiment will be described.
[0148] The X-ray imaging device of the sixth embodiment has the function of determining whether the subject 10 will fall from the upper plate 80 and sounding an alarm when the subject 10 is about to fall. Thus, the doctor can be warned that the subject 10 may fall from the upper plate 80 due to body movement caused by anesthesia to prevent the subject from falling.
[0149] like Fig.17 As shown, the X-ray imaging device of the sixth embodiment includes a shoulder / waist offset calculation unit 33 and a fall possibility determination unit 34 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments.
[0150] The camera image processing unit 20 of the X-ray imaging device of the sixth embodiment executes the Figure 5The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 After extracting the image of the surrounding area of the skeleton of the subject 10, execute Fig.18 The steps S120 to S122 are described in detail below.
[0151] (Steps S101 to S112)
[0152] The camera image processing unit 20 executes the first embodiment Figure 5 The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 By performing any of the actions in the process, the surrounding area of the subject's bones is extracted and displayed on the monitoring image display 15.
[0153] (Step S120)
[0154] The shoulder / waist offset calculation unit 33 reads the coordinates of the shoulder and waist of the subject 10 determined by the subject skeleton determination unit 25 and the coordinates of the shoulder and waist of the subject 10 determined for the previous frame from the memory 25a, and calculates the offset between frames. In addition, the object for calculating the offset is not limited to the previous frame, but may also be a frame that is a predetermined number of frames ago.
[0155] (Step S121)
[0156] The fall possibility determination unit 34 determines whether the displacement amount calculated in step S120 is equal to or larger than a predetermined threshold value, and issues an alarm if it is equal to or larger than the threshold value.
[0157] In the sixth embodiment, both the shoulder and waist offsets are calculated, but only one of them may be calculated.
[0158] Alternatively, the alarm may be emitted by display or light instead of sound.
[0159] <Implementation Method 7>
[0160] An X-ray imaging device according to the seventh embodiment will be described.
[0161] The X-ray imaging device of embodiment 7 determines which part of the subject is irradiated with X-rays based on the bones of the subject detected by any one of embodiments 1 to 3, and sets the irradiation conditions and image generation parameters to appropriate values corresponding to the part of the subject.
[0162] like Fig.19As shown, the X-ray imaging device of the seventh embodiment includes an X-ray irradiation region determination unit 35 and an X-ray irradiation condition / image generation parameter calculation unit 36 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments.
[0163] The camera image processing unit 20 of the X-ray imaging device of the seventh embodiment executes the Figure 5 Process and implementation method 2 Figure 6 The process and implementation method of 3 Figure 8 After extracting the image of the surrounding area of the skeleton of the subject 10, execute Fig. 20 The steps S123 to S126 are described in detail below.
[0164] (Steps S101 to S112)
[0165] The camera image processing unit 20 executes the first embodiment Figure 5 Process and implementation method 2 Figure 6 The process and implementation method of 3 Figure 8 By performing any of the actions in the process, the surrounding area of the subject's bones is extracted and displayed on the monitoring image display 15.
[0166] (Step S123)
[0167] The X-ray irradiation area determination unit 35 receives the focus detector distance (SII) from the X-ray irradiation device 11 and receives the movement amount from the X-ray tilt mechanism unit 61, the X-ray up and down movement mechanism unit 60 and the upper plate movement mechanism unit 82 to calculate the positional relationship between the upper plate 80 and the X-ray source in the X-ray irradiation device 11, and calculates the X-ray irradiation area in the image 21a.
[0168] (Step S124)
[0169] Furthermore, the X-ray irradiation area determination unit 35 receives the coordinates of the skeleton of the subject 10 from the subject skeleton determination unit 25, determines which part of the subject has entered the X-ray irradiation area, and obtains the X-ray irradiation part 211 (for example, the neck, chest, abdomen, waist, thigh, etc.) of the subject 10 (refer to Fig.21 ).
[0170] (Step S125)
[0171] The X-ray irradiation condition / image generation parameter calculation unit 36 refers to the internal table to calculate the X-ray irradiation conditions (eg, tube current / tube voltage) and image generation parameters suitable for the X-ray irradiation site 211 found in step S123 .
[0172] (Step S126)
[0173] The X-ray irradiation condition / image generation parameter calculation unit 36 sets the X-ray irradiation condition in the X-ray irradiation device 11 and sets the image generation parameter in the X-ray image generator 13 .
[0174] According to Embodiment 7, the part entering the irradiation field can be detected according to the position of the bones of the subject 10, and an appropriate X-ray dose can be irradiated, and a fluoroscopic image can be generated with appropriate parameter values. Therefore, halation of the fluoroscopic image can be prevented and the image quality can be improved.
[0175] <Implementation Method 8>
[0176] An X-ray imaging device according to the eighth embodiment will be described.
[0177] The X-ray imaging device of the eighth embodiment has a function of automatically moving the X-ray irradiation device 11 to a site designated by a doctor based on the skeleton of the subject detected in any one of the first to third embodiments.
[0178] like Fig. 22 As shown, the X-ray imaging device of the eighth embodiment includes an X-ray irradiation region determination unit 35 and a difference calculation unit 37 in the camera image processing unit 20 in addition to the configuration of the X-ray imaging device of the first to third embodiments.
[0179] The camera image processing unit 20 of the X-ray imaging device of the eighth embodiment executes the Figure 5 The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 After extracting the image of the surrounding area of the skeleton of the subject 10, execute Fig.23 The steps S127 to S131 are described in detail below.
[0180] (Steps S101 to S112)
[0181] The camera image processing unit 20 executes the first embodiment Figure 5 The process and implementation method of 2 Figure 6 The process and implementation method of 3 Figure 8 By performing any of the actions in the process, the surrounding area of the subject's bones is extracted and displayed on the monitoring image display 15.
[0182] (Step S127)
[0183] The X-ray irradiation area determination unit 35 receives the focus detector distance (SID) from the X-ray irradiation device 11 and receives the movement amount from the X-ray tilt mechanism unit 61, the X-ray vertical movement mechanism unit 60 and the upper plate movement mechanism unit 82 to calculate the positional relationship between the upper plate 80 and the X-ray source in the X-ray irradiation device 11, and calculates the X-ray irradiation area 240 (reference Fig.24 ).
[0184] (Step S128)
[0185] The difference calculation unit 37 receives the imaging designated part 242 (for example, the neck, chest, abdomen, waist, thigh, knee, stomach, etc.) of the subject 10 from the doctor.
[0186] (Step S129)
[0187] The difference calculation unit 37 receives the coordinates of the skeleton of the subject 10 from the subject skeleton determination unit 25, and calculates the coordinates of the imaging designated part 242 based on the coordinates of the skeleton of the subject in the image 21a (refer to Fig.24 ).
[0188] (Step S130)
[0189] The difference calculation unit 37 calculates the coordinate difference between the center 241 of the X-ray irradiation region 240 calculated in step S127 and the center of the imaging designated region 242 calculated in step S128 .
[0190] (Step S131)
[0191] The difference calculation unit 37 sets a movement amount corresponding to the difference calculated in step S125 for one or more of the X-ray tilt mechanism unit 61, the X-ray vertical movement mechanism unit 60, and the upper plate movement mechanism unit 82. As a result, the X-ray irradiation device 11 moves relative to the upper plate 80, and the center 241 of the X-ray irradiation area 240 coincides with the center of the designated imaging site 242.
[0192] According to the eighth embodiment, the doctor does not need to irradiate X-rays to capture fluoroscopic images and to move the X-ray irradiation device 11 relative to the upper plate 80 while observing the fluoroscopic images, thereby reducing X-ray exposure to the subject 10 and the doctor.
[0193] Furthermore, the doctor can save the trouble of positioning the subject 10 and the X-ray irradiation device 11 at a desired location.
Claims
1. An X-ray imaging device, characterized in that: have: an upper plate for carrying a subject; an X-ray irradiation device for irradiating the subject with X-rays; an X-ray detector for detecting X-rays irradiated from the X-ray irradiation device and transmitted through the subject; and an image processing unit for receiving an image from a camera photographing the subject and processing the image, The image processing unit detects skeletons of a plurality of human figures included in the image, extracts an image of the subject mounted on the upper plate from among the plurality of human figures based on the detected skeletons, and displays the extracted image of the subject on a connected display device.
2. The X-ray imaging device according to claim 1, characterized in that: The image processing unit determines that the human figure having the smallest angle between the center line of the skeleton of the human figure and the center line of the image is the image of the subject.
3. The X-ray imaging device according to claim 1, characterized in that: The image processing unit determines that the human figure having the largest distance between two shoulders of the skeleton of the human figure is the image of the subject.
4. The X-ray imaging device according to claim 1, wherein: The image processing unit determines as the image of the subject a human figure having at least one of the minimum distance L between the middle point of the line connecting the two shoulders of the skeleton of the human figure and the center of the image and the minimum angle δ formed by the line connecting the two shoulders and the horizontal line of the image.
5. The X-ray imaging device according to claim 1, characterized in that: The image processing unit calculates the movement amount of pixels in the surrounding area of the image of the subject and the pixels corresponding to the image acquired by the camera at a time point earlier than a predetermined time point than the time point of acquiring the image, and issues an alarm when the number of pixels whose movement amount exceeds a threshold is greater than a predetermined number.
6. The X-ray imaging device according to claim 1, characterized in that: The image processing unit further extracts a face region of the subject in the image and displays the extracted face region on a display device in an enlarged manner.
7. The X-ray imaging device according to claim 1, characterized in that: The image processing unit calculates an offset of at least one of the positions of the shoulder and the waist with respect to the image of the subject in the image and the image of the subject in the image acquired by the camera at a time point predetermined earlier than the time point of acquiring the image, and issues an alarm when the offset is greater than a predetermined threshold value.
8. The X-ray imaging device according to claim 1, characterized in that: The image processing unit determines a part of the subject to be irradiated with X-rays from the X-ray irradiation device based on the skeleton of the subject in the image, and sets X-ray irradiation conditions in the X-ray irradiation device according to the part.
9. The X-ray imaging device according to claim 1, characterized in that: Also available: a mechanism part, which moves the X-ray irradiation device relative to the upper plate, The image processing unit calculates the coordinates of the designated imaging part of the designated imaging part of the designated imaging part according to the skeleton of the object in the image, and calculates the coordinates of the center of the X-ray irradiation area in the image, and calculates the difference between the coordinates of the designated imaging part and the X-ray irradiation area, and sets a movement amount corresponding to the difference in the mechanism unit, sets the movement amount in the mechanism unit in a manner that the X-ray irradiation device moves relative to the upper plate, and moves the center of the X-ray irradiation area to the designated imaging part.
10. An image processing method for an X-ray imaging device provided with a camera, the image processing method being a method for processing an image captured by the camera of the X-ray imaging device provided with a camera, characterized in that: detecting skeletons of a plurality of human figures contained in an image captured by the camera, Based on the detected skeleton, an image of the subject mounted on the upper plate is extracted from the plurality of human images, and the extracted image of the subject is displayed on a connected display device.
Citation Information
Patent Citations
Radiographic system and control method thereof, and program
JP2021186199A
Skeleton detection system and work management device
JP2021196783A
