Non-contact respiratory measurement and visualization for chest X-ray imaging.

Abstract

The present invention relates to chest radiography. To improve image quality and consistency, a respiratory status determination device is provided, comprising an input unit, a processing unit, and an output unit. The input unit is configured to receive a sequence of depth images continuously captured with a sensor having a field of view covering a torso of a patient positioned for chest radiography imaging examination. The processing unit is configured to analyze the received sequence of depth images to determine changes in depth values ​​within one or more regions of interest (ROI) over time representative of respiratory movements of the patient, and to determine a respiratory signal based on the determined changes in depth values ​​inside the one or more ROIs over time. The output unit is configured to provide the determined respiratory signal.

Description

Technical Field

[0001] The present invention relates to chest X-ray imaging, and more particularly to a respiratory state determination device, a chest X-ray imaging system, a method for determining a patient's respiratory state, and a computer program element.

Background Art

[0002] Chest X-ray examination is one of the most important and frequent imaging diagnostic examinations. To obtain good radiation images, specific diagnostic requirements and image quality criteria are defined in international and domestic guidelines and recommendations. Chest X-ray imaging in the posterior-anterior (PA), anterior-posterior (AP), or lateral (LAT) projection should be performed "at full inspiration and at end-expiration". This is necessary for proper visualization and evaluation of the relevant lung structures in the radiation image.

[0003] In today's workflow, the operator usually gives a breathing command while being away from the examination room and then triggers X-ray image acquisition without visual feedback on the patient's actual respiratory state. In many cases, the patient does not fully follow the breathing instructions, for example, because the patient cannot understand the breathing command or because the patient cannot hold their breath long enough. The latter can occur particularly when the operator gives a breathing command while being in the examination room and it takes several seconds to reach the X-ray emission control.

[0004] However, X-ray imaging taken exactly at full inspiration facilitates the diagnostic task. Furthermore, images taken at the moment of strong breathing or other chest movements may be degraded by motion blur.

Summary of the Invention

Problems to be Solved by the Invention

[0005] There may be a need to improve the image quality.

Means for Solving the Problems

[0006] The object of the present invention is solved by the subject matter of the independent claims, and further embodiments are incorporated into the dependent claims. It should be noted that the embodiments of the present invention described below also apply to respiratory status determination devices, chest X-ray imaging systems, methods for determining a patient's respiratory status, and computer program elements.

[0007] According to a first aspect of the present invention, a respiratory state determination device is provided comprising an input unit, a processing unit, and an output unit. The input unit is configured to receive a sequence of depth images continuously captured by a sensor having a field of view covering the torso of a patient positioned for chest X-ray imaging. The processing unit is configured to analyze the received sequence of depth images to determine the change over time of depth values ​​within at least one region of interest (ROI) representing the movement of the patient's anatomical structures orthogonal to the image plane of the sensor and / or the movement of the patient's anatomical structures within the image plane of the sensor, and to determine a respiratory signal based on the change over time of depth values ​​inside the at least one ROI. The output unit is configured to provide the determined respiratory signal.

[0008] In other words, it is proposed to utilize a sequence of continuously acquired depth images to track changes in depth values ​​within one or more ROIs on the patient (e.g., on the patient's back, chest, and / or shoulders) in order to extract the patient's respiratory signal. The sequence of depth images includes multiple depth images arranged in chronological order. Each depth image may also be called an image data frame. The extracted respiratory signal may be used to guide or control the triggers for image acquisition in order to achieve different levels of automation.

[0009] Respiration alters the volume of a patient's thoracic cage, resulting in movement of different body parts, such as the thoracic / abdominal and shoulders, also known as respiratory motion. Depending on the patient's orientation (e.g., PA / AP or LAT), these motion vectors can be parallel or perpendicular to the image plane. This is particularly true for X-rays, in contrast to computed tomography (CT) and magnetic resonance (MR) images, for example, where there is no flat surface behind the patient. The respiratory state determination devices described herein are configured to capture respiratory motion components in the image plane, respiratory motion components perpendicular to the image plane, or both, based on depth camera signals within one or more ROIs, each ROI covering a specific patient anatomical structure (e.g., chest, shoulders, etc.).

[0010] For example, a sequence of depth images may be acquired by a sensor (e.g., a 3D camera) having an optical axis approximately perpendicular to the detector plane defined by the detector's front cover or bed. In this geometry setup, the image plane is equal to the detector plane. Furthermore, respiratory motion can manifest in two ways.

[0011] Local changes in depth values, i.e., the z-component of the sensor image (e.g., 3D camera image), resulting from the movement of the patient's back orthogonal to the detector plane (i.e., the image plane), and / or The up-and-down movement of the patient's shoulder, that is, the motion component in the image plane, that is, the y-component of the sensor image (i.e., the 3D camera image).

[0012] In this way, it is possible to determine the one-dimensional motion component to capture any change in the volume of the subject. It is not necessary to reconstruct a three-dimensional image of the patient's chest and generate real-time measurements of the patient's chest diameter to determine the patient's respiratory motion. Rather, a simple arithmetic mean of the depth values ​​inside at least one ROI, or the relative proportion of pixels representing the patient's anatomical structures inside the ROI, can be used to determine the patient's respiratory motion.

[0013] In one example, the respiratory signal may be displayed to the operator on the console and / or tube head to provide feedback on the actual respiratory status. The operator can check whether the patient is responding to the respiratory command and release X-ray exposure at the appropriate time.

[0014] In another example, respiratory signals may be further analyzed and classified, for example, using a recurrent neural network (RNN), and when deep inspiration and breath-holding are detected, acoustic and / or visual signals may be provided to the operator.

[0015] In a further example, automated gateding or triggering of image acquisition is performed based on real-time acquired respiratory data, allowing images to be acquired under predefined respiratory conditions such as complete inspiration. At this level of automation, the operator can enable image acquisition, for example, by pressing a button, but the system can delay actual image acquisition within a predefined time window to find the optimal timing in terms of respiratory state and patient (breathing) movement. System delays can be disabled by the user in response to clinical requirements, such as critical patient conditions.

[0016] In this way, there is no need to apply wearable sensors to the patient's body to monitor the patient's respiratory status during image acquisition. The patient enters the examination room "as is" and walks towards the desired target spot within it. Depth images of the patient's torso are captured, for example, by a range camera and then used to detect the movement of one or more ROIs on the patient, which can be used as indicators of the patient's respiratory signal. Depending on the view of the chest X-ray examination, one or more ROIs may be defined on the patient's chest, back (PA), chest (AP) shoulder, or an area covering both the patient's torso and the background, as described below, particularly with respect to Figures 3 to 7. Thus, the patient's respiratory signal can be derived from changes in depth values ​​within one or more ROIs and then used to guide or control the triggering of image acquisition to achieve different levels of automation.

[0017] As used herein, the term “image plane” refers to a plane perpendicular to the optical axis of a sensor (e.g., a depth camera) where a sharp image of an object point appears at least within the Gaussian optical system. In the case of a chest examination, the detector plane, i.e., the plane defined by the front cover or bed of the detector, is equal to the image plane when the camera is not tilted relative to the detector. In the case of an inclined view (not typically used in chest examinations), the detector plane may differ from the image plane. However, those skilled in the art will understand that the methods described herein can also be adapted to inclined views by inducing a nonlinear scaling of the respiratory signal.

[0018] In addition, respiratory signals provided by a respiratory status determination device can help determine the optimal time for X-ray emission. Patients can benefit from a reduction in the number of retakes due to the avoidance of motion blur, thereby promoting the ALARA ("reasonably achievable low As") objective. Radiologists can also enjoy better image quality for diagnostic tasks.

[0019] According to one embodiment of the present invention, at least one ROI includes at least two ROIs, including a first ROI and a second ROI. The processing unit is configured to determine the change in depth value inside the first ROI in time, which represents the movement of a first patient's anatomical structure orthogonal to the image plane, and to determine the change in depth value inside the second ROI in time, which represents the movement of a second patient's anatomical structure in the image plane. The processing unit is configured to determine a respiratory signal based on the determined change in depth value inside both the first and second ROIs.

[0020] In other words, by combining both motion components, including motion components orthogonal to the image plane and motion components within the image plane, a 1D signal representing the patient's respiratory signal can be generated. This will be explained in detail below, particularly with respect to the embodiment shown in Figure 4A.

[0021] According to one embodiment of the present invention, at least one ROI includes an ROI covering the patient's anatomical structure and the background on both sides of the patient.

[0022] Therefore, the rigid body movement does not change up to the total number of background pixels. Therefore, any overall patient movement within the image plane (often observed in the LAT view) does not change the ratio of patient pixels for a wide ROI that includes the background on both sides of the patient, i.e., does not change the average depth value of such an ROI. Therefore, determining the change in depth values within an ROI that includes the patient and the static background (e.g., averaging the depth values within the ROI or determining the ratio of patient pixels within the ROI) can capture the change in volume and is invariant to large-scale shifts of the object, i.e., robust to patient movement.

[0023] This is explained in detail below, particularly with respect to the examples shown in FIGS. 5A, 5B, 6A, 6B, 7A, and 7B.

[0024] According to one embodiment of the present invention, the processing unit is further configured to determine a time point for emitting x-ray exposure in a predetermined respiratory state based on the determined respiratory signal.

[0025] For example, the respiratory signal can be further analyzed and classified to detect, e.g., deep inspiration and breath-holding, and thus the time point for emitting x-ray exposure, using, for example, a recurrent neural network (RNN).

[0026] Automatic gating or triggering of image acquisition can be performed at the determined time point for emitting x-ray exposure.

[0027] In one example, when the patient is placed in the posterior-anterior (PA) position for a chest x-ray imaging examination, at least one ROI in the PA view includes an ROI of the patient's back.

[0028] In PA views, patient back movement perpendicular to the detector plane, i.e., the image plane, can lead to localized changes in depth values. Therefore, the patient's respiratory signal can be derived from changes in depth values ​​within the ROI of the patient's back.

[0029] In one example, when a patient is positioned in an anterior-posterior (AP) position for a chest X-ray, the AP view contains one or more ROIs on the patient's chest.

[0030] In AP views, patient chest movements perpendicular to the image plane can lead to localized changes in depth values. Therefore, the patient's respiratory signal can be derived from changes in depth values ​​within the ROI on the patient's chest.

[0031] According to embodiments of the present invention, when a patient is positioned in a lateral position for chest X-ray imaging, in the lateral (LAT) view, at least one ROI is

[0032] A first ROI includes pixels representing the patient's back and pixels representing areas within the background,

[0033] A second ROI having pixels representing the patient's chest and pixels representing a region within the background,

[0034] The third ROI on the patient's torso and It includes one or more of these.

[0035] The background can be defined by detectors, beds, system treatment tables, etc.

[0036] When a patient is positioned laterally for a chest X-ray, the patient's respiratory cycle can induce movement of the patient's back, i.e., a motion component in the image plane. This property can be used to extract respiratory signals from an ROI located near the patient's back. The set of depth values ​​inside the ROI is clearly separated into two clusters: one containing the patient's back and the other containing the background. The relative proportion of "patient pixels" inside the ROI has smaller depth values ​​than the background, changes with inspiration / expiration, and can therefore be used as an indicator of the patient's respiratory signal. A similar approach is applicable to an ROI that includes pixels representing the patient's chest and pixels representing areas within the background.

[0037] Another option is to place the ROI on the patient's torso without including pixels representing the background. During inspiration or expiration, the movement of the patient's torso perpendicular to the image plane (e.g., the detector plane) can lead to localized changes in depth values. Thus, the patient's respiratory signal can be derived from changes in depth values ​​within the ROI on the patient's torso. This will be explained in detail below, particularly with respect to the examples shown in Figures 6A, 6B, 6C, and 6D.

[0038] According to one embodiment of the present invention, at least one ROI includes an ROI having pixels representing a patient's shoulder and pixels representing an area in the background.

[0039] In PA, AP, and LAT views, respiratory motion can also manifest as the up-and-down movement of the patient's shoulder, i.e., as a motion component in the image plane. Therefore, the respiratory signal can be derived from an ROI located near the patient's shoulder in PA, AP, and LAT views. In depth images, the ROI around the detected shoulder landmark includes pixels representing the detector and pixels representing the patient. The set of depth values ​​inside the ROI is clearly separated into two clusters: one containing the patient and the other the background. The relative proportion of "patient pixels" inside the ROI has smaller depth values ​​than the background, changes with inspiration / expiration, and therefore can be used as an indicator of the patient's respiratory signal.

[0040] According to one embodiment of the present invention, at least one ROI includes an ROI that covers the patient's torso and background.

[0041] In a PA view, the ROI can cover both the patient's posterior chest and the background (e.g., the X-ray detector).

[0042] In AP views, the ROI can cover both the chest and the background (e.g., the X-ray detector or examination table).

[0043] In a LAT view, the ROI can cover the side of the chest and the background (e.g., the X-ray detector or examination table).

[0044] The respiratory signal estimated from such ROIs is robust to the rigid motion of the patient within the image plane. This is useful for solving the problem of proper macroscopic patient motion, which will be described below, particularly with respect to the embodiments shown in Figures 7A and 7B.

[0045] According to one embodiment of the present invention, the processing unit is configured to determine at least one ROI based on the active area of ​​an automatic exposure control (AEC) dosimetry chamber, or based on one or more anatomical landmarks of a patient.

[0046] In one example, when AEC is used for the majority of the examination, the X-ray technician must precisely position the chamber within the patient's lung area, so that this area closely matches the patient's lung location. This will be explained below, in particular with respect to the embodiments shown in Figures 2A and 2B.

[0047] In another example, anatomical landmarks can be used to define patient-adapted ROIs for respiratory analysis by using bounding boxes that encompass, for example, lung landmarks to be detected. This will be described below, in particular with respect to the embodiments shown in Figures 3A and 3B.

[0048] According to one embodiment of the present invention, the processing unit is configured to evaluate the change in the mean depth value inside at least one ROI.

[0049] For example, motion components orthogonal to the image plane (e.g., the movement of a patient's back in a PA view) result in reduced depth values ​​for "patient pixels." Therefore, the average depth value inside the ROI decreases.

[0050] For example, motion components in the image plane (e.g., chest movement in the LAT view or vertical shoulder movement in the PA view) result in an increased proportion of "patient pixels" compared to pixels representing the background. Consequently, the average depth value inside the ROI decreases.

[0051] The average depth value may be a weighted average or a regular average.

[0052] The average depth value represents a single one-dimensional (1D) time signal and can therefore be used as an indicator for the patient's respiratory movement.

[0053] According to a second aspect of the present invention, A chest X-ray imaging system, An X-ray imaging system comprising an X-ray source and an X-ray detector spaced apart from the X-ray source to accommodate a patient to be imaged, A sensor having a field of view covering the torso of a patient, positioned for the aforementioned chest X-ray imaging examination, and configured to continuously capture a sequence of depth images of the patient's torso, A respiratory state determination device according to any one of claims 1 to 9 and A chest X-ray imaging system having the following is provided.

[0054] For example, the sensor may include, for instance, a light detection and distance measurement (LIDAR), a radio detection and distance measurement (RADAR), or a three-dimensional (3D) non-contact motion scanner using a camera-based sensor. The 3D camera-based sensor may include, for example, a stereo-based sensor or an infrared video sensor.

[0055] In some examples, a single sensor, such as a range camera, is provided. In other examples, multiple sensors may be provided. For example, depth information may be derived from two or more separate cameras.

[0056] The sensor may be mounted in the X-ray system, for example, inside a tube head, or it may be removed from the system and mounted, for example, on the ceiling.

[0057] After the patient is positioned for chest X-ray imaging, the sensor (e.g., range camera) continuously acquires a series of depth image data frames to track changes in depth values ​​within one or more ROIs on the patient's back, chest, and / or shoulders during inspiration and expiration. Based on the changes in depth values ​​within one or more ROIs, the patient's respiratory signal can be established to guide or control the triggering of image acquisition by the X-ray imaging system.

[0058] The depth image sequence may be acquired by exposing the sensor to non-ionizing radiation. For example, infrared light may be used, but the use of light in the visible spectrum is also conceivable. The sensor may use a predefined structured light pattern projected onto the patient to perceive 3D image data. For example, the structured light pattern is a speckle pattern. For example, the sensor may be part of a range camera. Examples include Microsoft Kinect or ASUS Xtion Pro Live devices.

[0059] According to one embodiment of the present invention, the chest X-ray imaging system further comprises a feedback device configured to receive respiratory signals from a respiratory status determination device and provide feedback on the patient's respiratory status.

[0060] The feedback device may be an auditory feedback device (e.g., a speaker), a visual feedback device (e.g., a display), or an audiovisual feedback device.

[0061] For example, the respiratory signal may be displayed in real time on the operator console. The operator can then visualize the curve and select the optimal time for X-ray emission.

[0062] According to one embodiment of the present invention, the X-ray imaging system is configured to be manually controlled or automatically triggered for image acquisition at the time of emission of X-ray exposure determined by a respiratory state determination device.

[0063] In one example, respiratory signals are further analyzed and classified using, for example, a recurrent neural network (RNN), and when hold, deep inspiration, and breath-hold are detected, acoustic and / or visual signals are provided to the operator.

[0064] Furthermore, while the operator can enable image acquisition by, for example, pressing a button, the system can delay actual image acquisition within a predefined time window to find the optimal timing in relation to the respiratory state and patient (respiratory) movement. System delays can be disabled by the user in response to clinical requirements, such as critical patient conditions.

[0065] According to a third aspect of the present invention, A method for determining a patient's respiratory status in chest X-ray imaging, The steps include receiving a sequence of depth images continuously captured by a sensor having a field of view covering the torso of a patient, which is positioned for the aforementioned chest X-ray imaging examination, The steps include analyzing the sequence of received depth images to determine the change in depth values ​​over time within at least one region of interest (ROI) that represents the patient's respiratory movement, The steps include determining a respiratory signal based on the time-dependent change in the depth value inside the at least one ROI determined, The steps of providing the determined respiratory signal and A method is provided that has the following characteristics.

[0066] According to one embodiment of the present invention, the method further includes determining a time to release X-ray exposure under a predetermined respiratory state based on a determined respiratory signal.

[0067] According to another aspect of the present invention, a computer program element is provided for controlling a device according to the first aspect and any related example, or for controlling a system according to the second aspect and any related example, and is configured to perform a method according to the third aspect and any related example when executed by a processor.

[0068] These and other aspects of the present invention are evident from the embodiments described below and will be explained with reference thereto.

[0069] In the drawings, as in the literature, similar reference numerals generally refer to the same parts across different drawings. Furthermore, the drawings are not necessarily to scale, and the emphasis is on illustrating the principles of the present invention. [Brief explanation of the drawing]

[0070] [Figure 1] A chest X-ray imaging system and examples are shown. [Figure 2A] This shows the ROI on the patient's back, defined by the active area of ​​the automated exposure-controlled dosimetry chamber. [Figure 2B] The respiratory signal derived from the ROI shown in Figure 2A is shown. [Figure 3A] This shows the ROI of the patient's back, defined by the patient's anatomical landmarks. [Figure 3B] The respiratory signal derived from the ROI shown in Figure 3A is shown. [Figure 4A] This shows the ROI located near the patient's shoulder. [Figure 4B] The respiratory signal derived from the ROI shown in Figure 4A is shown. [Figure 5A] This shows an ROI that covers both the patient's posterior chest and background. [Figure 5B]The respiratory signal derived from the ROI shown in Figure 5A is shown. [Figure 6A] This shows different ROIs that jointly capture respiratory motion within the image plane and perpendicular to the image plane. [Figure 6B] The respiratory signal derived from the ROI shown in Figure 6A is shown. [Figure 7A] This shows different ROIs that jointly capture respiratory motion within the image plane and perpendicular to the image plane. [Figure 7B] Figure 7A shows the respiratory signal derived from the ROI, which occurs when rigid patient movement occurs. [Figure 8] This flowchart shows a method for determining a patient's respiratory status in chest X-ray imaging. [Modes for carrying out the invention]

[0071] Figure 1 schematically and illustratively shows one embodiment of the chest X-ray imaging system 100. The chest X-ray imaging system 100 comprises an X-ray imaging system 110, a sensor 120, and a respiratory state determination device 10.

[0072] The X-ray imaging system 110 includes an X-ray source 112 and an X-ray detector 114. The X-ray detector 114 is spaced apart from the X-ray source 112 to accommodate the patient PAT to be imaged.

[0073] Generally, during image acquisition, the collimated X-ray beam (indicated by arrow P) is emitted from the X-ray source 112, passes through the patient PAT in the region of interest (ROI), undergoes attenuation due to interaction with the material within it, and then the attenuated beam strikes the surface of the X-ray detector 114. The density of the organic material constituting the ROI determines the level of attenuation, such as the thoracic cavity and lung tissue in chest X-ray imaging. High-density materials (such as bone) cause higher attenuation than low-density materials (such as lung tissue). The digital values ​​to be registered for the X-rays are then integrated into an array of digital values ​​that form the X-ray projection image for a given acquisition time and projection direction.

[0074] The overall operation of the X-ray imaging system 110 can be controlled by an operator from a console 116. The console 116 may be coupled to a screen or monitor 118 on which acquired X-ray images or imager settings can be viewed or reviewed. An operator, such as a technician at a medical laboratory, can control image acquisition, which is performed by emitting individual X-ray exposures, via the console 116 by, for example, activating a joystick or pedal, or other appropriate input means coupled to the console 116.

[0075] In the example shown in Figure 1, the patient PAT is standing facing a flat surface with the X-ray detector 114 behind it. In another example (not shown), the X-ray imaging system 110 is C-arm type, and the patient PAT is actually lying on an examination table instead of standing.

[0076] The sensor 120 is configured to continuously capture a sequence of depth images of the patient's torso, which is positioned for chest X-ray imaging.

[0077] For example, sensor 120 may comprise a range camera having a projector that projects a cone of structured light onto the patient PAT. The cone of structured light is sometimes called the field of view (FOV). Examples of range cameras are Microsoft Kinect or ASUS Xtion Pro Live devices. The reflection of the light from the patient's surface to the camera is then recorded by sensor 120, which is also included in the camera. The “distortion” in the reflected speckle pattern is then recorded by comparing it to how the speckle pattern should have looked if the patient were not present. The recorded distortion is then converted for each pixel into a depth value, also called a distance value. It will be understood that the sensor and projector do not necessarily have to be located within the same camera housing as described above. In one example, the projector and sensor may be arranged as different components. However, it will be understood that range cameras can also operate according to different principles, such as time of flight, stereo triangulation, optical triangulation sheets, interferometers, and coded apertures.

[0078] In another example, the sensor 120 may comprise a stereo camera having two or more lenses, each having a separate image sensor or film frame. This allows the camera to simulate human binocular vision and thus gives it the ability to capture three-dimensional images in a process known as stereophotography.

[0079] In further examples, depth information may be derived from two or more separate cameras or from a laser scanner.

[0080] For example, other 3D non-contact motion scanners using LIDAR or radar may also be suitable for capturing a sequence of depth images of a patient's torso.

[0081] Depth image data can be fused with other sensor data. For example, depth camera information can be combined with optical flow from RGB camera channels, or other range data from, for example, an ultrasonic (US) sensor.

[0082] The sensor 120 can be mounted to enable proper monitoring of the patient's respiratory status.

[0083] In the example shown in Figure 1, the sensor 120 is mounted near the X-ray tube. In another example (not shown), the sensor 120 can be mounted near the focal point, on the ceiling of the examination room, or on the wall of the examination room.

[0084] The respiratory status determination device 10 can be any computing device, including desktop and laptop computers, smartphones, tablets, etc. The respiratory status determination device 10 may be a general-purpose device or a device with dedicated units suitable for providing the functions described later. In the example in Figure 1, the components of the respiratory status determination device 10 are shown integrated into a single unit. However, in alternative examples, some or all of the components may be configured as separate modules in a distributed architecture and connected by an appropriate communication network. The respiratory status determination device 10 and its components can be configured as a dedicated FPGA or as a hardwired standalone chip. In some examples, the respiratory status determination device 10 or some of its components may reside in a console 116 operating as software routines.

[0085] The respiratory state determination device 10 comprises an input unit 12, a processing unit 14, and an output unit 16. Each unit may be part of, or include, an application-specific integrated circuit (ASIC), electronic circuitry, a processor (shared, dedicated, or grouped) and / or memory (shared, dedicated, or grouped) running one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the functions described.

[0086] Broadly speaking, the sensor 120 is configured to acquire a sequence of depth images of the patient PAT upon exposure to non-ionizing radiation, such as visible light or infrared light. The sequence of depth images captures the patient's 3D shape or at least the patient's torso. Thus, the sequence of depth images "tracks" or describes changes in the outer surface or periphery of the patient PAT in 3D space for monitoring the patient's respiratory status.

[0087] Next, the sequence of depth images is supplied to the respiratory state determination device 10 via an input unit 12, which can be implemented as an Ethernet® interface, a USB® interface, a wireless interface such as WiFi® or Bluetooth®, or any equivalent data transfer interface that enables data transfer between an input peripheral and the processing unit 14. The processing unit 14 then processes the received sequence of depth images in a manner that will be described in more detail below and outputs a respiratory signal, also called a respiratory signal, via an output unit 16, which can be implemented as an Ethernet® interface, a USB® interface, a wireless interface such as WiFi® or Bluetooth®, or any equivalent data transfer interface that enables data transfer between an output peripheral and the processing unit 14.

[0088] The determined respiratory signal can be used to guide or control the trigger for image acquisition. The following examples illustrate different levels of automation.

[0089] In one example, the respiratory signal may be displayed to the operator on a screen 118 coupled to the console 116 and / or tube head to provide feedback on the actual respiratory status. The operator can verify whether the patient is responding to the respiratory command and, where appropriate, can announce X-ray exposure by activating a joystick or pedal, or other suitable input means coupled to the control panel 116.

[0090] In another example, the respiratory signal is further analyzed and classified by the respiratory state determination device 10, for example, using a recurrent neural network (RNN). Based on the respiratory data acquired in real time, visual or acoustic guidance may be provided to the operator to inform them about the respiratory state.

[0091] In a further example, automatic gateting or triggering of image acquisition is performed based on respiratory data acquired in real time, allowing images to be acquired under predefined respiratory conditions, such as complete inspiration. In this example, the operator can, for example, initiate an X-ray emission request, while the system checks the respiratory condition and releases exposure for image acquisition only if the patient is inhaling and holding their breath. The user can disable the automated gate and manually release exposure if there is a clinical need.

[0092] The following sections describe how to determine the respiratory signal from the image sequences registered by the sensor in PA, AP, and LAT views.

[0093] Chest X-ray in AP / PA projection Post-anterior (PA) refers to the direction of the X-ray beam's travel, i.e., the direction in which the X-ray beam strikes the posterior portion of the chest before the anterior portion. To acquire an image, the patient is asked to stand with their chest facing the X-ray detector, with their arms held above or to the sides, and their shoulders rolled forward. The X-ray technician may then ask the patient to take several deep breaths and hold them for a few seconds. This technique of holding the breath generally helps to obtain clear images of the heart and lungs in the image.

[0094] To obtain anterior-posterior (AP) images, the patient is asked to stand with their back to the X-ray detector. If the patient is unable to stand, AP images can be taken with the patient sitting or lying on their back on the bed.

[0095] In the following, the method is described for illustrative purposes for chest radiography in PA projection. However, those skilled in the art will understand that the method can also be adapted for chest radiography in AP projection. The background plane is defined by the detector, bed, system treatment table, etc. For ease of reading, objects in the background (e.g., the detector) are not shown.

[0096] Generally, respiratory movements are revealed in two ways.

[0097] This refers to the movement of the patient's back / chest perpendicular to the detector plane (or image plane), which results in a localized change in depth value, and the vertical movement of the patient's shoulders, i.e., the motion component within the image plane.

[0098] This disclosure proposes extracting one or both motion components from a sequence of depth images and obtaining a single 1D time signal from the extracted one or both motion components representing the patient's respiratory motion.

[0099] For example, a sequence of depth images may be acquired by a sensor (e.g., a 3D camera) having an optical axis substantially perpendicular to the detector plane defined by the detector's front cover or bed. In this geometry setting, it is equal to the image detector plane. Furthermore, respiratory motion is revealed in two ways.

[0100] The movement of the patient's back, perpendicular to the detector plane (i.e., the image plane), corresponds to a local change in depth value, i.e., the z component of the sensor image (e.g., a 3D camera image), while the up-and-down movement of the patient's shoulders, i.e., the motion component within the image plane, i.e., the y component of the sensor image (i.e., a 3D camera image). Details of the signal formation process will be described later.

[0101] In the first option, as shown in Figure 2A, one or more ROIs include the patient's back to extract the movement of the patient's back orthogonal to the image plane. In this example, the image plane is the detector plane. In the example in Figure 2A, the ROIs are defined based on system geometry parameters, more specifically by features representing the X-ray detector in the depth image. For example, an ROI for respiratory signal extraction may be defined by the active area of ​​an automated exposure control (AEC) dosimetry chamber, which can be projected onto the patient surface in the depth image. Since AECs are used for most examinations, the X-ray technician must precisely position the chamber within the patient's lung region, and thus this region should closely coincide with the patient's lung position.

[0102] Next, the processing unit 14 determines the change in depth value inside the ROI as a function of time representing the patient's respiratory movement, and determines the respiratory signal based on this. Figure 2B shows an example of a respiratory signal with deep inspiration and breath-holding extracted from the ROI shown in Figure 1A. In this example, the 1D respiratory signal is generated by averaging the depth value inside the ROI. Further processing with time filtering, such as a nonlinear filter or a Kalman filter, may be applied to suppress outliers and generate a smoother time signal. The X-ray image is acquired at t=0.

[0103] In the second option, as shown in Figure 3A, the ROI is defined by one or more anatomical landmarks of the patient in the depth image. For example, the processing unit 14 may be configured to estimate the location of external landmarks (e.g., shoulders) or internal landmarks (e.g., lung apex) from the depth image. For example, see "Evaluation of Collimation Prediction Based on Depth Image and Automated Landmark Detection for Routine Clinical Chest X-ray Examination" by J. Senegas, A. Saalbach, M. Bergtholdt, S. Jockel, D. Mentrup, R. Fischbach (In: AF Frangi et al. (eds.): MICCAI 2018, LNCS 11071, pp. 571 to 579, Springer, Cham (2018)). These landmarks may be used, for example, to define a patient-adapted ROI for respiratory analysis by using a bounding box that encompasses the detected lung landmarks. For example, in the case of a chest or chest X-ray, the hip and shoulder joints can be identified from the depth values ​​that make up the depth image, and the line connecting the two hip joints can be used as a lower boundary for the ROI. The line connecting the shoulder joints may be used as an upper boundary having two torso sides that form the lateral boundary.

[0104] Figure 3B shows an example of a respiratory signal with deep inspiration and breath-holding extracted from the ROI shown in Figure 3A. In this example, the 1D respiratory signal is also generated by averaging the depth values ​​inside the ROI. The X-ray image is acquired at t=0.

[0105] In the third option, as shown in Figure 4A, the patient's respiratory cycle can also induce vertical shoulder movement, i.e., a motion component in the image plane. This property can be used to extract respiratory signals from an ROI located near the patient's shoulder (indicated as ROI_shoulder). For example, a landmark detector algorithm can be used to detect the shoulder landmark. In the depth image, the ROI around the detected shoulder landmark includes pixels representing the detector and pixels representing the patient. The set of depth values ​​inside the ROI is clearly separated into two clusters: one containing the patient and the other containing the background (e.g., the X-ray detector or examination table).

[0106] Figure 4B shows an example of a respiratory signal with deep inspiration and breath-hold, extracted from the ROI shown in Figure 4A. The processing unit 14 is configured to average all depth values ​​inside the ROI, including the region around the detected shoulder landmark. During the upward shift of the shoulder due to inhalation, the relative proportion of “patient pixels” inside the ROI increases, although all have depth values ​​smaller than the detector. As a result, the average ROI depth value decreases. Similarly, during exhalation, the average ROI depth value increases. Thus, the average ROI depth value signal is an indicator of respiratory movement.

[0107] Figure 4A also shows an additional ROI (indicated as ROI_back) located on the patient's back region. The respiratory signals extracted from the additional ROI are shown in Figure 4B. In some cases, the two respiratory signals shown in Figure 4B can be combined to calculate the optimal respiratory signal.

[0108] Optionally, the vertical position of the transition between the patient region and the background region can be tracked. Inspiration results in an upward shift in the vertical position of this transition, while expiration results in a downward shift. Therefore, the vertical position of the shoulder / background transition constitutes a 1D signal representing the patient's respiratory movement.

[0109] As described above, respiratory motion can manifest in two types of localized image changes: a) motion components toward the sensor, i.e., motion components orthogonal to the image plane, such as the movement of the patient's back in the PA view or the movement of the patient's chest in the AP view, and b) motion components in the image plane, such as the up-and-down movement of the shoulders in the PA view or AP view.

[0110] As mentioned above, motion components orthogonal to the image plane can be determined according to the first and second options, while motion components within the image plane can be determined using a third option. Both motion components can be evaluated by averaging the depth pixels inside a appropriately selected local ROI and then combining them, for example, using a linear combination of the two signals. The average of all depth values ​​inside the ROI decreases as the patient inhales.

[0111] A fourth option allows for joint access to both motion components by selecting a large ROI (shown as ROI_large) that covers both the patient's torso and part of the background, i.e., both sides of the patient's background, as shown in Figure 5A. Figure 5A shows an example of a larger ROI. During the inspiratory phase, lung volume, and therefore torso volume, increases, increasing the number of “patient pixels” compared to background pixels, and decreasing the depth value of patient pixels.

[0112] As shown below, the relative change in the mean depth value inside the ROI after offset correction approximates the relative change in patient volume inside the ROI. Mean depth signal TIFF0007896633000001.tif77 is the average "patient thickness" TIFF0007896633000002.tif710, "Patient Pixel Count" N p It depends on the distance D from the sensor to the detector and the total number N of pixels inside the ROI. This will result in TIFF0007896633000003.tif10126.

[0113] Offset correction signal The relative change x(t) / x(0) of TIFF0007896633000004.tif9126 is the X-ray acquisition frame (t=0). Regarding TIFF0007896633000005.tif11126, Volume V p It is directly proportional to the relative change of .

[0114] Therefore, the relative change in the offset correction signal x approximates the change in the volume of the patient's torso. This corresponds to volume V. p average patient thickness TIFF0007896633000006.tif85 and, for example, patient area A using a rectangular prism model. p Let us assume that it can be approximated by the product of .

[0115] Figure 5B shows the respiratory signal derived from larger ROIs by averaging the depth values ​​inside the ROIs. The X-ray image is acquired at t=0.

[0116] For comparison, Figure 5A also shows the ROI on the patient's back (indicated as ROI_back), and Figure 5B shows the respiratory signal derived from it.

[0117] Respiratory signals from large ROIs are robust to rigid patient motion within the image plane, which will be described below, in particular with respect to the embodiments shown in Figures 7A and 7B.

[0118] Chest X-ray examination using LAT projection The approach described above may also be applicable to lateral protrusions, which may require additional sensors and / or different sensor locations, as described below.

[0119] Lateral views are typically taken to complement the anterior view. Lateral images, together with anterior images, are useful for locating lesions as they enable three-dimensional analysis. To obtain a lateral image, the patient is asked to rotate one shoulder onto a plate and raise their hand above their head. The technician may then ask the patient to take a deep breath and hold the position.

[0120] In the LAT view, one or more ROIs may comprise one or more of the ROIs shown in Figure 6A.

[0121] The first ROI (indicated as ROI_back) may comprise pixels representing the patient's back and pixels representing areas within the background (X-ray detector or examination bed). The set of depth values ​​inside the ROI is clearly separated into two clusters: the patient's back and the background. In this option, the processing unit 14 is configured to average all depth values ​​inside the ROI, including the area around the chest. During inhalation, the relative proportion of "patient pixels" inside the ROI increases, although all have smaller depth values ​​than the background. As a result, the average ROI depth value decreases. Similarly, during exhalation, the average ROI depth value increases. Thus, the average ROI depth value signal is an indicator of respiratory movement.

[0122] The second ROI (indicated as ROI_chest) may consist of pixels representing the patient's chest and pixels representing areas within the background. Similarly, the average ROI depth signal is an indicator of respiratory movement.

[0123] A third ROI (indicated as ROI_torso) may be placed on the patient's torso. The relative change in the mean depth value inside the ROI is an indicator of respiratory movement.

[0124] A fourth ROI (indicated as ROI_large) may be a large ROI that covers part of the patient's torso and background by extending beyond the boundary between the patient's chest and the background, and beyond the boundary between the patient's back and the background; that is, the fourth ROI covers the patient's torso and the background on both sides of the patient.

[0125] Figure 6B shows the respiratory signals derived from the depth values ​​inside these different ROIs. The X-ray images are acquired at t=0.

[0126] Respiratory signals from large ROIs are robust to rigid patient motion in the image plane, which frequently occurs in LAT examinations, as shown in Figures 7A and 7B. As can be seen from Figure 7B, respiratory signals derived from large ROIs (indicated as ROI_large) are robust to rigid patient motion in the image plane, while individual signals from ROIs on the patient's back and chest are corrupted by gradients due to patient motion.

[0127] As shown in Figures 2B to 7B, the determined respiratory signals are further analyzed by the processing unit 14, and based on the determined respiratory signals, the timing for releasing X-ray exposure under a predetermined respiratory state can be determined.

[0128] For example, a respiratory signal can be further analyzed and classified using a recurrent neural network (RNN). RNNs are typically a type of neural network applied to signals that have correlations between their values ​​over time. Essentially, an RNN is a loopback architecture of interconnected neurons and current inputs, where the last hidden state influences the output of the next hidden state. RNNs are ideally suited for sequential information and, because they also have memory, are well-suited for time-series data.

[0129] During training, different parameters of the network and layers are explored using a training dataset containing respiratory signals from multiple previously recorded patients. These previously recorded patient respiratory signals may be obtained from the same patient and / or from other patients. The time points for releasing X-ray exposure at given respiratory states within the dataset may be annotated by a specialist. After training, the RNN is used to extract temporal features, i.e., the time points for releasing X-ray exposure, from the received respiratory signals.

[0130] The X-ray imaging system can be manually controlled or automatically triggered for image acquisition at a predetermined time. For example, an operator can initiate an X-ray emission request, while the system checks the respiratory status and emits exposure only for image acquisition at the predetermined time. The user can disable the automated gate and emit exposure manually if there is a clinical need.

[0131] The respiratory status determination device 10 and chest X-ray imaging system 100 described above can be easily adapted to operate in a C-arm X-ray imaging device where the patient PAT lies on an examination table during image acquisition, rather than standing as shown in Figure 1.

[0132] Referring to Figure 8, a flowchart is shown illustrating the method for determining a patient's respiratory status in chest X-ray imaging.

[0133] In step 210, a sequence of depth images is received, which is continuously captured by a sensor having a field of view covering the torso of the patient positioned for chest X-ray imaging. Thus, once the patient is positioned for chest X-ray imaging, the sensor (e.g., a range camera) continues to acquire a sequence of depth image data frames of the patient's torso over an extended period. The acquired sequence of depth image data frames can be supplied via the input unit 12 to the respiratory state determination device 10 shown in Figure 1.

[0134] In step 220, the sequence of received depth images is analyzed to determine the change in depth values ​​over time within one or more regions of interest (ROIs) representing the patient's respiratory motion. In other words, depth values ​​within one or more ROIs may be recorded over individual sequences of depth images, and the change in depth values ​​may be established as a function of time.

[0135] In step 230, the respiratory signal is determined based on the change in depth values ​​within one or more ROIs over time. For example, the 1D respiratory signal may be generated by averaging the depth values ​​within one or more ROIs, using a weighted average or a normal average.

[0136] In step 240, the determined respiratory signal is output to a display, for example, on the console and / or tube head, providing feedback on the actual respiratory status. The operator can check whether the patient is responding to the respiratory command and release X-ray exposure at the appropriate time.

[0137] Optionally, the respiratory signals may be further analyzed and classified using, for example, a recurrent neural network (RNN).

[0138] In one example, when deep inhalation and breath-holding are detected, an acoustic and / or visual signal may be provided to the operator.

[0139] In one example, the timing for releasing X-ray exposure may be determined based on a respiratory signal, which determines the timing for releasing X-ray exposure at a given respiratory state. For example, an operator can initiate an X-ray emission request, while the system checks the respiratory state and releases exposure only for image acquisition at the determined time. The user can disable the automated gate and manually release the exposure if there is a clinical need.

[0140] In another exemplary embodiment of the present invention, a computer program or computer program element is provided, characterized in that it is adapted to perform a method step of a method according to one of the embodiments described above on a suitable system.

[0141] Accordingly, the computer program elements may be stored in a computer unit, which may be part of an embodiment of the present invention. This computing unit may be adapted to perform or trigger the execution of the steps of the method described above. Furthermore, it may be adapted to operate the components of the apparatus described above. The computing unit may be adapted to operate automatically and / or to execute user sequences. The computer program may be loaded into the working memory of a data processor. Accordingly, the data processor may be equipped to perform the method of the present invention.

[0142] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the outset and computer programs that, by means of updating, transform an existing program into a program that uses the present invention.

[0143] Furthermore, computer program elements can provide all the steps necessary to satisfy the procedure of the exemplary embodiment of the procedure described above.

[0144] According to a further exemplary embodiment of the present invention, a computer-readable medium such as a CD-ROM is presented, having computer program elements stored thereon, which are described in the previous section.

[0145] Computer programs may be stored and / or distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0146] However, computer programs may also be presented over a network such as the World Wide Web and downloaded from such a network into the working memory of a data processor. According to a further exemplary embodiment of the present invention, a medium is provided for making a computer program element available for download, and this computer program element is configured to perform a method according to one of the aforementioned embodiments of the present invention.

[0147] It should be noted that embodiments of the present invention are described with reference to different subject matter. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, unless otherwise notified, those skilled in the art will find that any combination of features belonging to one type of subject matter, as well as any combination of features relating to different subject matter, are gathered from the above and below descriptions and are deemed to be disclosed in this application. However, all features can be combined to provide a synergistic effect greater than the simple sum of the features.

[0148] Although the present invention is illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be considered illustrative or exemplary and not limiting. The present invention is not limited to the embodiments disclosed. Other variations of the disclosed embodiments can be understood and practiced by those skilled in the art in carrying out the claimed invention, based on a study of the drawings, disclosure and dependent claims.

[0149] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude plurals. A single processor or other unit may perform the functions of several items described in the claims. The mere fact that certain means are referenced in different dependent claims does not imply that combinations of these means cannot be used advantageously. No reference numeral in the claims should be construed as limiting in scope.

Claims

1. A respiratory status determination device, Input unit and Processing unit and Output unit and It has, The input unit is configured to receive a sequence of depth images continuously captured by a sensor having a field of view covering the torso of a patient positioned for chest X-ray imaging. The processing unit is configured to analyze the sequence of received depth images to determine over time the change in depth values ​​inside at least one region of interest (ROI) that represents the movement of the patient's anatomical structures orthogonal to the image plane of the sensor and / or the movement of the patient's anatomical structures within the image plane of the sensor, and to determine a respiratory signal based on the change in the average depth value inside the at least one ROI over time, wherein the image plane is a plane perpendicular to the optical axis of the sensor. The output unit is configured to provide the determined respiratory signal. Respiratory status determination device.

2. The aforementioned at least one ROI has at least two ROIs, including a first ROI and a second ROI. The processing unit is configured to determine over time the change in depth value inside the first ROI, which represents the movement of the anatomical structures of the first patient orthogonal to the image plane, and to determine over time the change in depth value inside the second ROI, which represents the movement of the anatomical structures of the second patient in the image plane. The processing unit is configured to determine the respiratory signal based on the change in the average depth value inside both the first and second ROIs. The respiratory state determination device according to claim 1.

3. The respiratory state determination device according to any one of claims 1 to 2, wherein the at least one ROI has an ROI that covers the anatomical structure of the patient and the background on both sides of the patient.

4. When the patient is positioned laterally for the chest X-ray imaging examination, the at least one ROI is A first ROI having pixels representing the patient's back and pixels representing an area within the background, A second ROI having pixels representing the patient's chest and pixels representing a region within the background, The third ROI of the patient's torso and A respiratory state determination device according to any one of claims 1 to 3, comprising:

5. The respiratory state determination device according to any one of claims 1 to 4, wherein the at least one ROI has an ROI having pixels representing the patient's shoulder and pixels representing an area in the background.

6. The respiratory state determination device according to any one of claims 1 to 5, wherein the at least one ROI has an ROI that covers the patient's torso and background.

7. The respiratory state determination device according to any one of claims 1 to 6, wherein the processing unit is configured to determine the at least one ROI based on the active area of ​​an automatic exposure control (AEC) dosimetry chamber, or based on one or more anatomical landmarks of the patient.

8. The respiratory state determination device according to any one of claims 1 to 7, wherein the processing unit is further configured to determine the timing for emitting X-ray exposure in a predetermined respiratory state based on the determined respiratory signal.

9. A chest X-ray imaging system, An X-ray imaging system comprising an X-ray source and an X-ray detector spaced apart from the X-ray source to accommodate a patient to be imaged, A sensor having a field of view covering the torso of a patient, positioned for the aforementioned chest X-ray imaging examination, and configured to continuously capture a sequence of depth images of the patient's torso, A respiratory state determination device according to any one of claims 1 to 8 and A chest X-ray imaging system having the following features.

10. A feedback device configured to receive respiratory signals from the respiratory state determination device and provide feedback regarding the patient's respiratory state. The chest X-ray imaging system according to claim 9, further comprising the above.

11. The chest X-ray imaging system according to claim 9 or 10, wherein the X-ray imaging system is configured to be automatically triggered or manually controlled for image acquisition at the time for emitting X-ray exposure determined by the respiratory state determination device.

12. A method for determining a patient's respiratory status in chest X-ray imaging, The steps include receiving a sequence of depth images continuously captured by a sensor having a field of view covering the torso of a patient, which is positioned for the aforementioned chest X-ray imaging examination, A step of analyzing the sequence of received depth images to determine over time the change in depth values ​​inside at least one region of interest (ROI) that represents the movement of the patient's anatomical structures orthogonal to the image plane of the sensor and / or the movement of the patient's anatomical structures within the image plane of the sensor, wherein the image plane is a plane perpendicular to the optical axis of the sensor. The steps include determining the respiratory signal based on the change over time of the mean depth value inside the at least one ROI, The steps of providing the determined respiratory signal and A method having

13. Based on the determined respiratory signal, the step of determining the timing for releasing X-ray exposure under a predetermined respiratory state. The method according to claim 12, further comprising the above.

14. A computer program for controlling the apparatus described in any one of claims 1 to 8, or for controlling the system described in any one of claims 9 to 11, wherein when executed by a processor, the computer program is configured to perform the method described in claim 12 or 13.

Images