Patient monitoring during scanning

Abstract

An imaging system is provided for capturing images of a patient during a medical scan using a medical scanner having a scanning system and a patient support table. A camera is fixed relative to the scanning system and captures images of the patient support table. Within the captured images, the direction in which the patient support table moves corresponds to a fixed vector direction. In response to patient support table motion, a motion field calculation is performed between sequentially captured images. Regions of the motion field that are parallel to the fixed vector direction (and thus substantially free of motion perpendicular to the fixed vector direction) are identified as patient positioning features. This information can be utilized in a number of ways as part of an automated workflow solution.

Description

Technical Field

[0001] This invention relates to monitoring patients during medical scanning, and more particularly to distinguishing patient setup from other nearby objects (e.g., operators and static components of the scanner). Background Technology

[0002] In many medical imaging procedures, monitoring patients during the imaging process is crucial, such as during the preparation phase for guided and automated examinations, monitoring patient movement before and during the procedure, or monitoring the patient's health status. For example, monitoring images (e.g., video images provided by a camera such as a wide-field-of-view camera) are used for this monitoring.

[0003] Imaging can also be used to monitor the workflow of the scanning process.

[0004] Camera-based workflow solutions require algorithmic approaches to effectively and accurately filter relevant signals from background signals. Typically, camera-based workflow solutions consist of a complex integration of nested AI neural network detection algorithms. Ideally, all neural network algorithms are thoroughly trained to achieve stable detection performance.

[0005] Given the complexity of the examination workflow, separating patient placement from the rest of the scenario is one of the most important and challenging tasks.

[0006] One specific application requiring precise separation is collision detection. In this application, it's necessary to detect parts of the patient preparation scene that might collide with the scanning gantry bore due to patient support table movement, before the patient is moved to the target scanning position. In such applications, limiting the number of false positives while maintaining a low false negative rate is crucial to avoid generating a large number of unnecessary alarms. Therefore, in this task, only those parts affected by patient support table movement are relevant.

[0007] Separating these parts based solely on static camera data (RGB or depth) is not an easy task.

[0008] Therefore, there is a need for an imaging system that can distinguish patient positioning from other objects with low computational complexity. Summary of the Invention

[0009] This invention is defined by the claims.

[0010] According to an example of one aspect of the present invention, an imaging system is provided for capturing images of a patient during a medical scan using a medical scanner having a scanning system and a patient support table, the imaging system comprising: A camera, which is mounted in a fixed position relative to the scanning system and facing the patient support table to capture images; and Processor, wherein the processor is configured to: The direction of movement of the patient support table is assigned as a fixed vector direction within the image captured by the camera; In response to patient support table movement, motion field calculations are performed between sequentially captured images, including calculating motion fields in motion directions parallel to the fixed vector direction and motion fields in motion directions perpendicular to the fixed vector direction; Identify image regions where the motion field is parallel to the direction of the fixed vector; and The identified image regions are output as identifiers of patient positioning features.

[0011] This imaging system identifies patient positioning features by performing a simple motion field analysis between sequential images. This analysis is triggered, for example, by signals from sensors on the patient support table. It provides an effective separation of the patient positioning from static elements such as nearby people, objects, and the scanner. Specifically, the background is stationary, the patient positioning moves along a predetermined direction, while the operator moves randomly, resulting in a discontinuous motion field. Separating these scene elements is not straightforward but offers significant opportunities for improving scene understanding, particularly for safety-related features in workflow camera solutions. It also provides a means to significantly improve the robustness of detection against background noise. In medical imaging applications involving controlled patient support table movement, it is known that patient support table movement creates a precisely defined motion field along a given motion vector.

[0012] The term "patient positioning characteristics" is intended to refer to all objects and items that move with the patient support table.

[0013] In the patient support table coordinate system, the patient's position is a very stable part of the scene. Projecting the actual measurement flow of the entire scene onto the patient support table's motion vector during its movement allows for a clear depiction and segmentation of the patient's position, including all auxiliary components moving along the table's motion vector. This scalar signal can then be used to filter the actual full-scene motion field.

[0014] The filtered motion field can then be used additionally as a pseudo-stereo signal to derive 3D information, but only within the relevant patient positioning volume. Specifically, the motion signal between two frames can be used to generate 3D information: for each pixel, the disparity equals the flow vector. This disparity value is combined with the known tabletop motion to derive depth information. Alternatively, 3D information can also be provided independently by the camera itself (if it is a 3D camera). In this case, the two depth values ​​can be combined to improve robustness or accuracy.

[0015] As mentioned above, the derived 3D information is uniquely selective, and therefore it is often difficult to generate using only typical depth sensors (such as time-of-flight cameras or stereo cameras).

[0016] For example, the camera is configured to be positioned such that the row direction of the captured images is aligned with the direction of movement of the patient support table. This simplifies the calculation of the motion field.

[0017] Alternatively, the processor can be configured to transform the camera images so that the row orientation of the transformed images is aligned with the direction of movement of the patient support table. This requires additional image processing but simplifies system installation.

[0018] In one example, the processor includes an input for receiving patient support table motion information from the medical scanner, thereby detecting the patient support table motion. In another example, the processor is also configured to analyze camera images to derive patient support table motion information, thereby detecting the patient support table motion. In this way, the system can be a standalone system, without requiring input from the medical scanner.

[0019] The processor can also be configured to generate a region mask for the patient positioning. This mask can be used to distinguish between objects that will move within the scanner chamber and those that will not, allowing the system to focus on the relevant image portion. This can improve detection rates, for example, by better distinguishing between patients and objects, or by better distinguishing between attachments attached to the patient and attachments not attached to the patient.

[0020] The processor can also be configured to calculate a depth signal for patient positioning. Stream analysis enables depth estimation, thereby distinguishing different objects. The camera may include a 3D camera, and the depth signal obtained from the depth camera can be combined with the calculated depth signal. This can be used to improve accuracy and / or robustness.

[0021] The processor can also be configured to perform patient-related calculations or calculations related to patient positioning.

[0022] For example, the processor can also be configured to perform proximity calculations between the patient positioning features and the scanning system for collision avoidance. Thus, scanner bore collision avoidance can be implemented as a safety feature. If a collision is detected or predicted, the system can automatically slow down the speed of the patient support table to allow the system operator to make adjustments. Therefore, the movement of the patient support table can be controlled based on proximity calculations.

[0023] The present invention also provides a medical scanner, comprising: Scanning system; The patient support table extends through the scanning system; A drive system for driving the patient support stage through the scanning system; and The imaging system as defined above.

[0024] For example, the camera of the imaging system is mounted relative to the scanning system in such an orientation that the row direction of the captured images is aligned with the direction of movement of the patient support table. This simplifies flow analysis.

[0025] The present invention also provides an image processing method for processing patient images during a medical scan using a medical scanner, the medical scanner having a scanning system, a patient support table, and a camera, the camera being mounted at a fixed position relative to the scanning system and facing the patient support table to capture images, wherein the method includes: The direction of movement of the patient support table is assigned as a fixed vector direction within the image captured by the camera; In response to patient support table movement, motion field calculations are performed between sequentially captured images, including calculating motion fields in motion directions parallel to the fixed vector direction and motion fields in motion directions perpendicular to the fixed vector direction; Identify image regions where the motion field is parallel to the direction of the fixed vector; and The identified image regions are output as identifiers of patient positioning features.

[0026] This method is based on simple image analysis to detect areas that remain static in the patient support table reference frame, thereby detecting patient positioning features.

[0027] This method may include transforming the camera image such that the row orientation of the transformed image is aligned with the direction of movement of the patient support table. This allows the camera to be mounted in any orientation relative to the patient support table.

[0028] The method may include receiving patient support table motion information from the medical scanner to detect the patient support table motion, or analyzing camera images to derive patient support table motion information, thereby detecting the patient support table motion.

[0029] The method may also include: Generate the area mask for the patient positioning; and / or Calculate the depth signal of the patient positioning; and / or Perform calculations related to the patient or the patient's positioning.

[0030] The present invention also provides a computer program including computer program code, which, when run on a processor of an imaging system as defined above, is adapted to implement the above-described method.

[0031] These and other aspects of the invention will become apparent and will be illustrated with reference to one or more embodiments described below. Attached Figure Description

[0032] To better understand the invention and to more clearly illustrate how to implement it, reference will now be made to the accompanying drawings by way of example only, in which: Figure 1 An example of a medical scanner is shown schematically; Figure 2 The results of the motion analysis are shown; and Figure 3 The image processing method is shown. Detailed Implementation

[0033] The invention will be described with reference to the accompanying drawings.

[0034] It should be understood that while the detailed description and specific examples indicate exemplary embodiments of the apparatus, system, and method, they are intended for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, system, and method of the present invention will be better understood from the following description, claims, and drawings. It should be understood that the drawings are schematic only and are not drawn to scale. It should also be understood that the same reference numerals are used in all drawings to indicate the same or similar parts.

[0035] This invention provides an imaging system for capturing images of a patient during a medical scan using a medical scanner having a scanning system and a patient support table. The camera is fixed relative to the scanning system and captures images of the patient support table. Within the captured images, the direction of movement of the patient support table corresponds to a fixed vector direction. In response to the movement of the patient support table, a motion field is calculated between sequentially captured images. Regions of the motion field parallel to this fixed vector direction are identified as patient positioning characteristics. Such regions substantially have no movement perpendicular to this fixed vector direction. That is, there may be no movement in the vertical direction, or the amount of movement is below a threshold. This information can be utilized in various ways as part of an automated workflow solution.

[0036] Figure 1 An example of a medical scanner 100 is schematically shown, which is used to acquire medical images of an object, and also includes an imaging system for capturing optical images of the patient during the medical scan. These images are generated for the purpose of determining patient movement.

[0037] The medical scanner 100 includes a scanning system, such as a CT imaging system 140, adapted to acquire medical images of a patient 121 positioned on a patient support table 120, i.e., CT images in this example. The patient support table 120 is adapted to move the patient 121 through the CT imaging system 140 during the CT imaging process. For this purpose, the medical scanner has a drive system 124 for driving the patient support table through the scanning system.

[0038] The imaging system includes an optical camera 130 and a processor 150. The optical camera is adapted to acquire monitoring images of the patient 121 during a CT imaging procedure. The operation of a system with a single camera will be explained, but multiple cameras are also possible. The camera can be a color or monochrome camera. It can use visible or infrared light. The camera can be a 2D imaging camera or a depth camera. As described below, depth information can be obtained from the depth camera or by analyzing 2D images using the patient's support table movement as additional input.

[0039] Camera 130 preferably has a wide field of view, such that it captures a full view of the patient support table 120, or at least the portion of the patient support table 120 where the area of ​​interest for the patient will be placed. Furthermore, movement of the patient support table 120 necessitates an even wider field of view so that the desired portion of the patient support table remains within the field of view of the camera (which is statically mounted relative to the scanning system, e.g., mounted on the ceiling) throughout the entire medical scan.

[0040] For example, the camera includes a fisheye lens with a field of view greater than 150 degrees, such as 160 degrees or more. Therefore, the patient support table (and the patient on the patient support table) can be imaged by a single camera or a camera group consisting of a small number of cameras, as shown in the figure.

[0041] In one example, the camera view has no perspective distortion relative to the patient support table, so that the movement of the patient support table is strictly along a predefined vector direction, such as along the image x-axis.

[0042] However, in another instance, any such distortion is corrected. For example, a wide-angle lens can cause image distortion, resulting in objects appearing differently in different areas of the camera's field of view. To address this issue, processor 150 can perform image post-processing to correct those distortions within the captured image caused by the camera's field of view width. This is shown as post-processing unit 160. For post-processing, the camera is calibrated such that geometric distortions are corrected through post-processing, and the camera's position and orientation relative to the scanner coordinate system are known.

[0043] The camera 130 is mounted in a fixed position relative to the static components of the scanner. The resulting camera view (or distortion-corrected camera image) ensures that the movement of the patient support table is strictly along a predefined vector direction.

[0044] The camera's position and orientation are calibrated relative to the scanner's patient support platform. This is known as the calculation of extrinsic parameters, or camera pose parameters. Based on the calibration, the exact pixel orientation of the patient support platform's motion is known.

[0045] Preferably, the camera image is transformed so that the patient support table moves strictly along, for example, the x-direction of the image.

[0046] Figure 2 Typical results of motion field calculations are shown. Optical flow calculation is one suitable example of motion field calculation. For simplicity, motion field calculation will be referred to as "flow calculation" below. The image above shows a camera view of patient 121 on patient support table 120.

[0047] The lower left image shows the flow calculation results along the flow direction parallel to the fixed vector direction (i.e., along the direction of movement of the patient support table), and the lower right image shows the flow calculation results along the flow direction perpendicular to the fixed vector direction.

[0048] As shown in the figure, the patient support table exhibits flow (i.e., movement) in the parallel direction but not in the vertical direction. Therefore, the movement of the patient support table is parallel to the vector direction. The system operator is visible in both flow calculation images. The operator moves near the patient support table, for example, by simultaneously pressing the patient support table move button to move the patient support table into position. Therefore, only the patient positioning area (including all components connected to the patient and the patient support table) has a clear x-direction flow (indicating x-direction movement). In particular, the entire patient positioning area has a strong x-flow component while completely (or almost completely) lacking a y-flow component, while the operator area exhibits a mixture of y and x flows, indicating discontinuous movement.

[0049] For the operator area, the absence of the y-flow component (or more generally, the y-flow component being below the threshold level) is extremely unlikely, but it is very likely for patient positioning, as this is the most stable part of the scenario.

[0050] The stream computing results exhibit subpixel sensitivity. Besides clearly distinguishing operator and patient positioning, no spurious signals are generated from any background objects (such as monitor stands, rack displays, or the magnets themselves). This demonstrates the signal's strong capability for its intended purpose of separating patient positioning from the background.

[0051] When a patient support table movement event is reported or detected, stream computing is performed between consecutive frames. Image regions that represent patient positioning features are then identified.

[0052] The camera should be able to measure at a sufficient frame rate to follow the patient support table's movement, ideally within millimeters between frames. The system either has a connection to the scanner system's software and hardware control unit to obtain information about the patient support table's movement, or, if it is a separate system without a connection, has its own AI-based system to detect the patient support table and movement events. Therefore, the medical scanner either reports patient support table movement through a separate system hardware component or detects it using a dedicated algorithm.

[0053] The detection and quantification of patient support table motion events immediately triggers the execution of stream computing, enabling a short detection latency, such as 50ms at a 20Hz frame rate.

[0054] The identification of patient positioning features can be used to visually inform the user, or it can be used in a machine-readable form to control medical scanners or other devices.

[0055] This information can be used, for example, to generate a region mask for patient positioning, including all connecting parts that move with the positioning. This mask distinguishes between objects that will move within the scanner chamber and those that will not, thus allowing the system to focus on the relevant portion of the image.

[0056] A depth signal for patient positioning can also be generated, including all connecting components that move with the positioning. This signal can then be highly selective and used to calculate or refine patient-related calculations (e.g., isocenter, posture) or patient positioning-related calculations (e.g., bore proximity calculation). Bore proximity calculation, for example, is used for collision detection to detect unsafe patient positioning conditions.

[0057] This area mask is used, for example, to select signals from other sensors for further analysis, such as selecting depth sensor signals of interest.

[0058] Patient positioning is observed by a camera typically mounted above the patient support table. For 3D cameras, video images (including color and depth) are analyzed to detect the patient and patient attachments. This image analysis is then used to check whether the positioning is complete, appropriate, and safe.

[0059] For 2D cameras, depth information can be obtained based on the analysis of 2D images and incorporating information about the movement of the patient support table. Such methods for obtaining depth information are known, for example, from WO 2019 / 076734. This document discloses a system and method for determining height contours from patient images taken on a moving patient support table.

[0060] All objects and devices within the camera's field of view are analyzed. Some objects will be located on the patient support table and will move into the borehole along with the table: these typically include the patient, coils, positioning supports, and physiological sensing devices. Other objects will not move with the patient support table: these include the operator, suspended injection equipment, physiological monitors, displays, etc.

[0061] The ability to distinguish between objects and parts that will move into the bore allows image processing to focus on relevant parts while ignoring others. This typically improves detection rates, for example, by better differentiating patients from other objects and by better distinguishing attachments attached to the patient from those not attached.

[0062] For collision detection with the scanner bore, the system should be able to detect components that will collide with the bore due to patient support table displacement before a collision occurs. Clearly, only objects moving with the scanning table are relevant to this analysis. By improving this distinction, the number of false alarms in the collision detection system is reduced.

[0063] In the event of a detected or predicted collision, the patient support table can automatically decelerate so that the operator can take action.

[0064] As described above, the system preferably utilizes depth data, especially to make collision detection robust. The depth data can come from a depth sensor (e.g., a time-of-flight sensor), from streaming analysis of conventional (RGB) images, or from a combination of both.

[0065] Note that collision detection is particularly challenging in image-guided therapy patient positioning scenarios, as these environments typically involve very crowded and busy workspaces containing numerous peripheral devices or their sub-components, as well as a large number of people standing nearby. This invention is extremely useful for ensuring the safe rotation of the detector, thereby avoiding collisions with patient positioning.

[0066] This invention is applicable to any medical scanner that moves the patient support table during the imaging process, such as PET imaging devices, MR imaging devices, SPECT imaging devices, and CT scanners as described above. The medical scanner may include a C-arm or a closed scanner bore. One or more cameras may be located inside the bore or on the inner surface of the C-arm, or outside the bore or C-arm enclosure. However, in all cases, the camera is stationary relative to the body of the scanning system, and therefore the patient support table moves relative to the camera. The camera may be mounted in the room where the scanner is located or directly onto the medical scanner.

[0067] Although in the above embodiments, the patient support table is always a patient support table in which the patient is lying down during the acquisition of medical images, the patient support table can also be configured for patients in a sitting or standing position.

[0068] Figure 3 An image processing method is shown for processing patient images during medical scanning using a medical scanner with a scanning system and a camera.

[0069] In step 200, the direction of movement of the patient support table is assigned as a fixed vector direction within the image captured by the camera.

[0070] In step 202, in response to the movement of the patient support table, motion field calculations are performed between the sequentially captured images, including calculating the motion field in the motion direction parallel to the fixed vector direction and the motion field in the motion direction perpendicular to the fixed vector direction.

[0071] In step 204, image regions where the motion is parallel to the fixed vector direction are identified.

[0072] In step 206, the identified image region is output as the identification of patient positioning features.

[0073] The processor assigns the direction of movement of the patient support table to a fixed vector direction within the image. This fixed vector direction can be a predefined direction, such as the x-direction of the image. This provides clear x-direction movement for the patient positioning area (including all components connected to the patient and the patient support table). As mentioned above and in the example, the camera image may have perspective distortion. Any such distortion can be corrected through image post-processing, after which the direction of movement of the patient support table is assigned to a fixed vector direction within the image.

[0074] A fixed vector direction is assigned to the patient support table movement within the image, and the motion field is calculated between sequentially captured images. The motion field calculation is performed between sequentially captured images, including motion directions parallel to and perpendicular to the fixed vector direction. Motion field calculation between consecutive images relative to a fixed reference direction provides the flow or motion direction relative to that fixed direction. The patient support table will have motion parallel to the fixed vector direction, while the system operator will have motion both parallel and perpendicular to that vector direction. Therefore, the operator's motion will be discontinuous, a mixture of parallel and perpendicular motion components. Consequently, the patient positioning area (including all components connected to the patient and the patient support table) will only have a motion field along the direction parallel to the fixed direction. Image regions where the motion field is parallel to the fixed vector direction are identified, and the identified image regions are output as identification of patient positioning features. Identified image regions will have no motion field in the direction perpendicular to the fixed vector direction, or the motion field will be below a threshold.

[0075] By studying the accompanying drawings, the disclosure, and the claims, those skilled in the art can understand and implement variations of the disclosed embodiments when carrying out the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the quantifiers "a" or "an" do not exclude a plurality.

[0076] The functions implemented by a processor can be implemented by a single processor or by multiple separate processing units, which together can be considered to constitute a "processor". These processing units can be located far apart from each other in some cases and communicate with each other via wired or wireless means.

[0077] The fact that certain measures are described in mutually different dependent claims does not mean that a combination of these measures cannot be used advantageously.

[0078] Computer programs can be stored / distributed on suitable media, such as optical storage media or solid-state media, which are provided with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0079] If the term “suitable” is used in the claims or description, it should be noted that the term “suitable” is intended to be equivalent to the term “configured as.” If the term “arrangement” is used in the claims or description, it should be noted that the term “arrangement” is intended to be equivalent to the term “system,” and vice versa.

[0080] Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. An imaging system for capturing images of a patient during a medical scan using a medical scanner having a scanning system (140) and a patient support table (120), said imaging system comprising: A camera (130) is mounted in a fixed position relative to the scanning system and facing the patient support table to capture images; as well as Processor (150), wherein the processor is configured to: (200) Assign the direction of movement of the patient support table to a fixed vector direction within the image captured by the camera; (202) In response to the movement of the patient support table, perform motion field calculations between sequentially captured images, including calculating motion fields in motion directions parallel to the fixed vector direction and motion fields in motion directions perpendicular to the fixed vector direction; (204) Identify image regions of the motion field that are parallel to the direction of the fixed vector; and (206) Output the identified image region as an identifier of the patient's positioning features.

2. The imaging system according to claim 1, wherein: The camera (130) is configured to be positioned such that the row direction of the captured image is aligned with the direction of movement of the patient support table; or The processor (150) is configured to transform the camera image such that the row direction of the transformed image is aligned with the direction of movement of the patient support table.

3. The imaging system according to any one of claims 1 to 2, wherein: The processor (150) includes an input for receiving patient support table motion information from the medical scanner and thereby detecting the patient support table motion; or The processor (150) is also configured to analyze camera images to derive patient support table motion information, thereby detecting the motion of the patient support table.

4. The imaging system of any of claims 1 to 3, wherein, The processor (150) is also configured to generate a region mask of the patient's position based on the movement of the patient support table.

5. The imaging system of any of claims 1 to 4, wherein, The processor (150) is also configured to calculate the depth signal of the patient's positioning.

6. The imaging system of claim 5, wherein, The camera includes a 3D camera, and wherein a depth signal obtained from the depth camera is combined with a calculated depth signal.

7. The imaging system of any of claims 1 to 6, wherein, The processor (150) is also configured to perform patient-related calculations or patient positioning-related calculations.

8. The imaging system of any of claims 1 to 7, wherein, The processor (150) is also configured to perform proximity calculations between the patient positioning features and the scanning system for collision prevention, and optionally to control the movement of the patient support table based on the proximity calculations.

9. A medical scanner, comprising: Scanning system (140); A patient support table (120) extends through the scanning system; Drive system (124) for driving the patient support stage through the scanning system; and The imaging system according to any one of claims 1 to 8.

10. The medical scanner of claim 9, wherein, The camera (130) of the imaging system is mounted relative to the scanning system in such an orientation that the row direction of the captured image is aligned with the direction of movement of the patient support table.

11. An image processing method for processing images of a patient during a medical scan using a medical scanner having a scanning system, a patient support table, and a camera mounted in a fixed position relative to the scanning system and facing the patient support table to capture images, wherein, The method includes: (200) Assign the direction of movement of the patient support table to a fixed vector direction within the image captured by the camera; (202) In response to the movement of the patient support table, perform motion field calculations between sequentially captured images, including calculating motion fields in motion directions parallel to the fixed vector direction and motion fields in motion directions perpendicular to the fixed vector direction; (204) Identify image regions of the motion field that are parallel to the direction of the fixed vector; and (206) Output the identified image region as an identifier of the patient's positioning features.

12. The method of claim 11, further comprising transforming the camera image such that the row direction of the transformed image is aligned with the direction of movement of the patient support table.

13. The method according to claim 11 or 12, comprising receiving patient support table motion information from the medical scanner to detect the patient support table motion, or analyzing camera images to derive patient support table motion information, thereby detecting the patient support table motion.

14. The method according to any one of claims 11 to 13, further comprising: Generate the area mask for the patient's positioning; and / or Calculate the depth signal of the patient positioning; and / or Perform calculations related to the patient or the patient's positioning.

15. A computer program comprising computer program code, wherein when the program is run on a processor of an imaging system according to any one of claims 1 to 10, the computer program code is adapted to implement the method according to any one of claims 11 to 14.

Images