Patient posture detection device
Millimeter-wave and terahertz sensors, combined with visible or infrared sensors and processing units, address the challenge of patient occlusion during medical imaging, ensuring accurate patient positioning and landmark detection, thus enhancing scanning efficiency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- KONINKLIJKE PHILIPS NV
- Filing Date
- 2023-10-16
- Publication Date
- 2026-04-22
AI Technical Summary
Existing medical image scanning and patient preparation methods are time-consuming and error-prone, particularly when using optical sensors, as they struggle to accurately localize anatomical key points or landmarks due to patient occlusion, such as being covered by a blanket.
Employing millimeter-wave and/or terahertz sensors to acquire images of patients on medical imaging units, combined with visible or infrared sensors, and utilizing a processing unit to determine patient posture, even when partially shielded, through image data analysis and neural networks.
Enables accurate patient positioning for medical imaging by overcoming occlusion, enhancing image resolution, and improving the determination of anatomical landmarks, thereby streamlining clinical workflows.
Smart Images
Figure 2026512941000001_ABST
Abstract
Description
Technical Field
[0004]
[0001] The present invention relates to a patient posture detection device, a patient posture detection system, a patient posture detection method, a computer program, and a computer-readable medium.
Background Art
[0002] Medical image scanning, examinations, or patient preparation for examinations are time-consuming and error-prone tasks. To simplify and accelerate the clinical workflow, automation of this part of the examination using an optical sensor (camera) is an effective option. One of the important requirements for such a system is the very accurate localization of the patient's anatomical key points or landmarks, which are used, for example, to automatically determine the scan position, the patient's posture (toe / head first, supine / prone, etc.), or for safety functions such as collision detection.
Summary of the Invention
Problems to be Solved by the Invention
[0003] When relying on an optical sensor, blocking the patient can complicate the localization of anatomical key points or landmarks. FIG. 1 shows an example of such a case where a blanket is placed over the patient, making it impossible to accurately localize the key points or landmarks of the lower limbs. It is necessary to address these problems. US Patent Application Publication No. 2019 / 0143145 (A1) discloses a system and method for patient monitoring for radiation therapy. US Patent Application Publication No. 2020 / 281539 discloses the creation of a digital twin for health diagnosis. US Patent Application Publication No. 2007 / 014391 discloses a system and method for treating a patient using radiation.
[0004] It would be beneficial to have improved techniques to assist in obtaining the patient's posture prior to medical image scanning.
Means for Solving the Problems
[0005] The object of the present invention is resolved by the subject matter of the independent claim, and other embodiments are incorporated into the dependent claims.
[0006] In a first embodiment, a patient posture detection device is provided, comprising at least one millimeter-wave (hereinafter referred to as "millimeter wave") sensor and / or a terahertz sensor, and a processing unit.
[0007] At least one millimeter-wave sensor and / or terahertz sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. At least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. At least one millimeter-wave sensor and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to a processing unit. The processing unit is configured to determine the patient's posture on the patient support of the medical imaging unit. Determining the patient's posture on the patient support includes utilizing the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0008] Thus, millimeter-wave and / or terahertz radiation, which can propagate through various materials, makes it possible to determine the patient's position on a patient support such as a bed, even when the patient is shielded, such as when the patient is partially covered with a blanket. This ensures that the patient can be accurately positioned for a medical imaging scan or examination within a medical imaging unit such as a CT scanner, MRI scanner, or PET scanner before being placed inside the medical imaging unit.
[0009] It is possible to use either millimeter-wave detection only (30-300 GHz) or terahertz detection only (300-3000 GHz), and it should be noted that both modalities allow for the acquisition of images with occlusion. However, by using both millimeter-wave and terahertz sensors, the image bandwidth can be effectively increased.
[0010] In one example, at least one millimeter-wave sensor and / or terahertz sensor is configured to move to multiple locations to acquire millimeter-wave image data of a patient on a patient support of a medical imaging unit.
[0011] In this way, by moving one or more millimeter-wave sensors and / or terahertz sensors, higher-resolution millimeter-wave images of the patient can be obtained, improving the ability to accurately determine the patient's posture.
[0012] In one example, at least one millimeter-wave sensor and / or terahertz sensor comprises a plurality of millimeter-wave sensors and / or terahertz sensors spaced apart from each other.
[0013] By using multiple millimeter-wave and / or terahertz sensors to view the patient from different locations and therefore from different angles, higher-resolution millimeter-wave and / or terahertz images of the patient can be obtained, improving the ability to accurately determine the patient's posture.
[0014] In one example, the device includes a visible or infrared sensor. The visible or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The visible or infrared sensor is configured to provide the visible or infrared image data of the patient on the patient support of the medical imaging unit to a processing unit. Determining the patient's posture on the patient support may include utilizing the visible or infrared image data of the patient on the patient support of the medical imaging unit.
[0015] In this way, high-resolution visible or infrared images can be used to enhance millimeter-wave and / or terahertz images, allowing for a better determination of the patient's posture.
[0016] In one example, the millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame, and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame following the first time frame. The visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit, which is acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0017] This allows for the acquisition of a first millimeter-wave image and a second millimeter-wave image, or a first terahertz image and a second terahertz image. When both millimeter-wave and terahertz sensors are used, they acquire images simultaneously, and these simultaneous images can be effectively merged to form a first millimeter-wave / terahertz image, which can then be merged again during subsequent acquisitions to form a second millimeter-wave / terahertz image.
[0018] In this way, the first millimeter-wave and / or terahertz image and associated visible image can be acquired before the patient is partially shielded, for example, before a blanket is placed over part of the patient's body. Next, the unshielded visible or infrared image and the associated millimeter-wave and / or terahertz image can be used together to provide information on how the millimeter-wave and / or terahertz image can be better interpreted with respect to the patient's posture. Then, the patient may be at least partially shielded, for example, by a blanket being placed over part of their body. Subsequent millimeter-wave and / or terahertz images are acquired, which can be seen through the shield (seen through the blanket). Next, the information derived from the unshielded visible or infrared image and the associated millimeter-wave and / or terahertz image can be used to better interpret the millimeter-wave and / or terahertz image of the shielded patient, and to accurately determine the patient's posture.
[0019] In one example, the processing unit is configured to determine the locations of multiple landmarks of the patient in a first time frame. Determining the locations of multiple landmarks of the patient in the first time frame involves using a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible or infrared image of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the locations of multiple landmarks of the patient in a second time frame. Determining the locations of multiple landmarks of the patient in the second time frame involves using the determined locations of multiple landmarks of the patient in the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support may include using the determined locations of multiple landmarks of the patient in the second time frame.
[0020] In other words, the first millimeter-wave and / or terahertz image and associated visible or infrared image are used to determine the location of landmark features such as the patient's ankles, knees, hips, wrists, elbows, shoulders, and head, and the high-resolution visible / infrared image helps in locating the landmarks within the millimeter-wave and / or terahertz image. Then, in practice, this combined millimeter-wave (terahertz) / visible or infrared image and its landmarks can be used to help locate the same landmarks in the patient in subsequent millimeter-wave images of the patient, in this case the patient is partially occluded. This facilitates the accurate determination of the patient's posture when the patient is partially occluded.
[0021] In one example, visible or infrared image data of a patient on the patient support of the medical imaging unit includes a second visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0022] In one example, determining the location of multiple patient landmarks in a second time frame involves using a second visible or infrared image of the patient on a patient support of a medical imaging unit.
[0023] In this way, visible / infrared and millimeter-wave and / or terahertz image data are used in combination both before and after the patient is partially shielded (e.g., a blanket covers part of the patient's body), allowing the patient's posture to be determined when the patient's posture is partially shielded.
[0024] In one example, determining the patient's posture on a patient support involves the use of a trained neural network.
[0025] In one example, the trained neural network is trained based on a plurality of image pairs of one or more reference persons. The plurality of image pairs includes millimeter wave and / or terahertz images of one or more reference persons and related visible or infrared images of one or more persons acquired simultaneously with the millimeter wave and / or terahertz images of one or more reference persons.
[0026] In one example, the plurality of image pairs of one or more reference persons includes image data of one or more reference persons having different degrees of body shielding with respect to visible or infrared image data.
[0027] Thus, the neural network is trained using both millimeter wave images and visible / infrared images of persons who are not shielded in both images of the first image set and are shielded to various degrees in both images of other image sets. Thereby, the neural network can receive millimeter wave and visible / infrared image pairs of unshielded patients and subsequent millimeter wave and visible / infrared image pairs of shielded patients, whereby the posture of the shielded patients can be accurately determined.
[0028] In a second aspect, a patient posture detection system is provided, which has a medical imaging unit, at least one millimeter wave sensor and / or terahertz sensor, and a processing unit.
[0029] At least one millimeter-wave sensor and / or terahertz sensor is arranged in the vicinity of a patient on a patient support of a medical imaging unit. The at least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before medical image data of the patient is acquired by the medical imaging unit. The at least one millimeter-wave sensor and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to a processing unit. The processing unit is configured to determine the posture of the patient on the patient support of the medical imaging unit. The determination of the posture of the patient on the patient support includes the use of the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0030] In a third aspect, there is provided a patient posture detection method, including steps of: before medical image data of a patient is acquired by a medical imaging unit, acquiring millimeter-wave and / or terahertz image data of the patient on a patient support of the medical imaging unit by at least one millimeter-wave sensor and / or terahertz sensor located in the vicinity of the patient on the patient support of the medical imaging unit; providing, by the at least one millimeter-wave sensor and / or terahertz sensor, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to a processing unit; and determining, by the processing unit, the posture of the patient on the patient support of the medical imaging unit, wherein determining the posture of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0031] In one aspect, there is provided a computer program for controlling an apparatus according to the first aspect, which is configured to execute the method according to the third aspect when executed by a processor.
[0032] In one embodiment, a computer program for controlling a system according to a second embodiment is provided, which is configured to perform the method of a third embodiment when executed by a processor.
[0033] Accordingly, according to some embodiments, a computer program is provided that controls one or more of the aforementioned devices / systems, which is adapted to perform the aforementioned method when the computer program is executed by a processor.
[0034] In another embodiment, a computer-readable medium storing the aforementioned computer program is provided.
[0035] The computer program may be, for example, a software program, but it may also be an FPGA, PLD, or any other suitable digital means.
[0036] Conveniently, the advantages provided by any of the above embodiments apply equally to all the other embodiments, and vice versa.
[0037] The above aspects and embodiments will become apparent from and be explained with reference to the embodiments described below. [Brief explanation of the drawing]
[0038] [Figure 1] This figure shows a patient preparation scene before imaging using a medical imaging unit, illustrating that the patient is partially obscured by a blanket, making it extremely difficult to pinpoint the position of the lower limbs. [Figure 2] A diagram showing an example of a patient posture detection device. [Figure 3] A diagram showing an example of a patient posture detection system. [Figure 4] A diagram illustrating a method for detecting patient posture. [Figure 5] A diagram illustrating the workflow of a detailed embodiment of a patient posture detection device, system, and method. [Modes for carrying out the invention]
[0039] The following describes exemplary embodiments with reference to the attached drawings.
[0040] Figure 2 shows an example of a patient posture detection device 10 having at least one millimeter-wave (hereinafter referred to as "millimeter-wave") sensor and / or terahertz sensor 20 and a processing unit 30. The at least one millimeter-wave sensor and / or terahertz sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The at least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The at least one millimeter-wave sensor and / or terahertz sensor is configured to provide the processing unit with the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the patient's posture on the patient support of the medical imaging unit. Determining the patient's posture on the patient support involves using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0041] Thus, millimeter-wave and / or terahertz radiation, which can propagate through various materials, makes it possible to determine the patient's position on a patient support such as a bed, even when the patient is shielded, such as when partially covered with a blanket. This ensures that the patient can be accurately positioned for a medical imaging scan or examination within a medical imaging unit such as a CT scanner, MRI scanner, or PET scanner before being placed inside the medical imaging unit.
[0042] It is possible to use either millimeter-wave detection in the 30-300 GHz range or terahertz detection in the 300-3000 GHz range, but it should be noted that both modalities allow for the acquisition of images with occlusion. However, by using both millimeter-wave and terahertz sensors, the image bandwidth can be effectively increased.
[0043] In one example, the processing unit is configured to control at least one millimeter-wave sensor and / or terahertz sensor to acquire millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data.
[0044] For example, at least one millimeter-wave sensor and / or terahertz sensor is configured to move to multiple locations to acquire millimeter-wave image data of a patient on a patient support of a medical imaging unit.
[0045] In this way, by moving one or more millimeter-wave sensors and / or terahertz sensors, higher-resolution millimeter-wave images of the patient can be obtained, improving the ability to accurately determine the patient's posture.
[0046] For example, at least one millimeter-wave sensor and / or terahertz sensor comprises a plurality of millimeter-wave sensors and / or terahertz sensors spaced apart from each other.
[0047] By using multiple millimeter-wave and / or terahertz sensors to view the patient from different locations and therefore from different angles, higher-resolution millimeter-wave and / or terahertz images of the patient can be obtained, allowing for accurate determination of the patient's posture.
[0048] For example, the device includes a visible or infrared sensor. The visible or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The visible or infrared sensor is configured to provide the visible or infrared image data of the patient on the patient support of the medical imaging unit to a processing unit. Determining the patient's posture on the patient support may include using the visible or infrared image data of the patient on the patient support of the medical imaging unit.
[0049] In this way, high-resolution visible or infrared images can be used to enhance millimeter-wave and / or terahertz images, allowing for a better determination of the patient's posture.
[0050] In one example, the processing unit is configured to control a visible or infrared sensor to acquire visible or infrared image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires the patient's medical image data of the patient.
[0051] For example, the millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame, and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame following the first time frame. The visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0052] This allows for the acquisition of a first millimeter-wave image and a second millimeter-wave image, or a first terahertz image and a second terahertz image. When both millimeter-wave and terahertz sensors are used, they acquire images simultaneously, and these simultaneous images can be effectively merged to form a first millimeter-wave / terahertz image, which can then be merged again during subsequent acquisitions to form a second millimeter-wave / terahertz image.
[0053] In this way, the first millimeter-wave and / or terahertz image and associated visible image can be acquired before the patient is partially shielded, for example, before a blanket is placed over part of the patient's body. Next, the unshielded visible or infrared image and the associated millimeter-wave and / or terahertz image can be used together to provide information on how the millimeter-wave and / or terahertz image can be better interpreted with respect to the patient's posture. Then, the patient may be at least partially shielded, for example, by a blanket being placed over part of their body. Subsequent millimeter-wave and / or terahertz images are acquired, which can be seen through the shield (seen through the blanket). Next, the information derived from the unshielded visible or infrared image and the associated millimeter-wave and / or terahertz image can be used to better interpret the millimeter-wave and / or terahertz image of the shielded patient, and to accurately determine the patient's posture.
[0054] For example, the processing unit is configured to determine the locations of multiple landmarks of the patient in a first time frame. Determining the locations of multiple landmarks of the patient in the first time frame involves using a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible or infrared image of the patient on the patient support of the medical imaging unit, and the processing unit is configured to determine the locations of multiple landmarks of the patient in a second time frame. Determining the locations of multiple landmarks of the patient in the second time frame involves using the determined locations of multiple landmarks of the patient in the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support may include using the determined locations of multiple landmarks of the patient in the second time frame.
[0055] In other words, the first millimeter-wave and / or terahertz image and associated visible or infrared image are used to determine the location of landmark features such as the patient's ankles, knees, hips, wrists, elbows, shoulders, and head, and the high-resolution visible / infrared image helps in locating the landmarks within the millimeter-wave and / or terahertz image. Then, in practice, this combined millimeter-wave (terahertz) / visible or infrared image and its landmarks can be used to help locate the same landmarks in the patient in subsequent millimeter-wave images of the patient, in this case the patient is partially occluded. This facilitates the accurate determination of the patient's posture when the patient is partially occluded.
[0056] For example, visible or infrared image data of a patient on a patient support of a medical imaging unit includes a second visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0057] For example, determining the positions of multiple patient landmarks in a second time frame involves using a second visible or infrared image of the patient on a patient support of a medical imaging unit.
[0058] In this way, visible / infrared and millimeter-wave and / or terahertz image data are used in combination both before and after the patient is partially shielded (e.g., a blanket covers part of the patient's body), allowing the patient's posture to be determined when the patient is partially shielded.
[0059] For example, determining the patient's posture on a patient support involves the use of a trained neural network.
[0060] For example, a trained neural network is trained on multiple image pairs of one or more reference individuals. These image pairs include millimeter-wave and / or terahertz images of one or more reference individuals, and associated visible or infrared images of one or more individuals acquired simultaneously with the millimeter-wave and / or terahertz images of the reference individuals.
[0061] In one example, multiple image pairs are reference images of a person on a patient support. Therefore, it is possible to train a neural network using images of a person who is not on a patient support and who may actually be in a completely different scene, and such a neural network will function well for patient support scenes. However, the reference image pairs may include several image pairs of a person on a patient support.
[0062] For example, multiple image pairs of one or more reference persons include image data of one or more reference persons having different degrees of body occlusion relative to visible or infrared image data.
[0063] Therefore, the neural network is trained using both millimeter-wave and visible / infrared images of a person who is unoccluded in both images of the first image set and occluded to varying degrees in both images of a further image set. This allows the neural network to receive millimeter-wave and visible / infrared image pairs for unoccluded patients and subsequent millimeter-wave and visible / infrared image pairs for occluded patients, from which it can accurately determine the pose of the occluded patient.
[0064] Figure 3 shows an example of a patient posture detection system 100 comprising a medical imaging unit 110, at least one millimeter-wave sensor and / or terahertz sensor 20, and a processing unit 30. At least one millimeter-wave sensor and / or terahertz sensor is positioned near the patient on the patient support of the medical imaging unit. At least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires medical image data of the patient. At least one millimeter-wave sensor and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. The processing unit is configured to determine the posture of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support involves utilizing the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0065] In one example, the processing unit is configured to control at least one millimeter-wave sensor and / or terahertz sensor to acquire millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data.
[0066] In one example, at least one millimeter-wave sensor and / or terahertz sensor is configured to move to multiple locations to acquire millimeter-wave and / or terahertz image data of a patient on a patient support of a medical imaging unit.
[0067] In one example, at least one millimeter-wave sensor and / or terahertz sensor comprises a plurality of millimeter-wave sensors and / or terahertz sensors spaced apart from each other.
[0068] In one example, the system includes a visible or infrared sensor. The visible or infrared sensor is positioned near the patient on the patient support of the medical imaging unit. The visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The visible or infrared sensor is configured to provide the visible or infrared image data of the patient on the patient support of the medical imaging unit to a processing unit. Determining the patient's posture on the patient support may include utilizing the visible or infrared image data of the patient on the patient support of the medical imaging unit.
[0069] In one example, the processing unit is configured to control a visible or infrared sensor to acquire visible or infrared image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires the patient's medical image data of the patient.
[0070] In one example, the millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame, and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame following the first time frame. The visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0071] In one example, the processing unit is configured to determine the locations of multiple landmarks of the patient in a first time frame. Determining the locations of multiple landmarks of the patient in the first time frame involves using a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible or infrared image of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the locations of multiple landmarks of the patient in a second time frame, and determining the locations of multiple landmarks of the patient in the second time frame involves using the determined locations of multiple landmarks of the patient in the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support involves using the determined locations of multiple landmarks of the patient in the second time frame.
[0072] In one example, visible or infrared image data of a patient on the patient support of the medical imaging unit includes a second visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0073] In one example, determining the location of multiple patient landmarks in a second time frame involves using a second visible or infrared image of the patient on a patient support of a medical imaging unit.
[0074] In one example, determining the patient's posture on a patient support involves the use of a trained neural network.
[0075] In one example, a trained neural network is trained on multiple image pairs of one or more reference individuals. These image pairs include millimeter-wave and / or terahertz images of one or more reference individuals, and associated visible or infrared images of one or more reference individuals acquired simultaneously with the millimeter-wave and / or terahertz images.
[0076] In one example, multiple image pairs of one or more reference individuals include image data of one or more reference individuals having different degrees of body occlusion relative to visible or infrared image data.
[0077] Figure 4 shows a patient posture detection method 200, which includes: acquiring millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit using at least one millimeter-wave sensor and / or terahertz sensor located near the patient on the patient support of the medical imaging unit before the medical image data of the patient is acquired by the medical imaging unit (210); providing the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to a processing unit using at least one millimeter-wave sensor and / or terahertz sensor (220); and determining the posture of the patient on the patient support of the medical imaging unit by the processing unit (230), wherein determining the posture of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.
[0078] In one example, the method includes controlling at least one millimeter-wave sensor and / or terahertz sensor by a processing unit to acquire millimeter-wave and / or terahertz image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires the patient's medical image data.
[0079] In one example, the method includes moving at least one millimeter-wave sensor and / or terahertz sensor to multiple locations to acquire millimeter-wave and / or terahertz image data of a patient on a patient support of a medical imaging unit.
[0080] In one example, at least one millimeter-wave sensor and / or terahertz sensor comprises a plurality of millimeter-wave sensors and / or terahertz sensors spaced apart from each other.
[0081] In one example, the system has a visible or infrared sensor, which is positioned near the patient on the patient support of the medical imaging unit. The method includes acquiring visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The method includes providing the visible or infrared image data of the patient on the patient support of the medical imaging unit to a processing unit using the visible or infrared sensor. Determining the patient's posture on the patient support may include utilizing the visible or infrared image data of the patient on the patient support of the medical imaging unit.
[0082] In one example, the method includes controlling a visible or infrared sensor by a processing unit to acquire visible or infrared image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires the patient's medical image data.
[0083] In one example, the millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame, and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame following the first time frame. The visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0084] In one example, the method includes a processing unit determining the locations of multiple landmarks of the patient in a first time frame. Determining the locations of multiple landmarks of the patient in the first time frame includes using a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible or infrared image of the patient on the patient support of the medical imaging unit. The method also includes a processing unit determining the locations of multiple landmarks of the patient in a second time frame. Determining the locations of multiple landmarks of the patient in the second time frame includes using the determined locations of multiple landmarks of the patient in the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support may include using the determined locations of multiple landmarks of the patient in the second time frame.
[0085] In one example, visible or infrared image data of a patient on the patient support of the medical imaging unit includes a second visible or infrared image of the patient on the patient support of the medical imaging unit, acquired simultaneously with a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
[0086] In one example, determining the locations of multiple patient landmarks in a second time frame involves utilizing a second visible or infrared image of the patient on a patient support of a medical imaging unit.
[0087] In one example, determining the patient's posture on a patient support involves utilizing a trained neural network.
[0088] In one example, a trained neural network is trained on multiple image pairs of one or more reference individuals. These image pairs include millimeter-wave and / or terahertz images of one or more reference individuals, and associated visible or infrared images of one or more reference individuals acquired simultaneously with the millimeter-wave and / or terahertz images.
[0089] In one example, multiple image pairs of one or more reference individuals include image data of one or more reference individuals having different degrees of body occlusion relative to visible or infrared image data.
[0090] Referring to Figure 5, the patient posture detection device, system, and method will be described in more detail. The description focuses on millimeter-wave radiation, but is also applicable to terahertz radiation.
[0091] The inventors of this invention have discovered that millimeter-wave radiation, sometimes called millimeter-RF radiation, can be used in entirely novel ways to enable the determination of a patient's posture in preparation for a medical imaging scan or examination / test. For example, millimeter-wave sensors or scanners have been developed for detecting shielded objects for airport security. For example, see: D. McMakin et al "New Improvements to Millimeter-Wave Body Scanners", Proceedings of 3DBODY.TECH 2017, 8th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Montreal, Canada, 11-12 Oct. 2017, pages 263-271; D. Li et al "Multi-Person Action Recognition in Microwave Sensors", Oral Session H1: Emerging Multimedia Applications, MM '20, October 12-16, 2020, Seattle, WA, USA, pages 411-420; R. Feger et al "A 77-GHz FMCW MIMO Radar Based on an SiGe Single-Chip Transceiver", IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, VOL. 57, NO. 5, MAY 2009, pages 1020-1035; F. Garcia-Rial et al "Combining Commercially Available Active and Passive Sensors Into a Millimeter-Wave Imager for Concealed Weapon Detection", IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, VOL. 67, NO.3, MARCH 2019, pages 1167-1183; Y. Lu et al "Study on feasibility of remote metal detection using millimeter wave radar for convenient and efficient security check", CCF Transactions on Pervasive Computing and Interaction (2021) 3, pages 284-299; G. Tzydynzhapov et al "New Real-Time Sub-Terahertz Security Body Scanner", Journal of Infrared, Millimeter, and Terahertz Waves, 41, 2020, pages 632-641; RZ Syeda et al "Sparse MIMO Array for Improved 3D mm-Wave Imaging Radar", Proceedings of the 17th European Radar Conference, 2021, pages 342-345.
[0092] Figure 5 shows a workflow related to a detailed embodiment of an apparatus, system, and method for determining the posture of a occluded patient using visible and millimeter-wave images acquired before and after partial occluding of the patient. However, in one embodiment, the millimeter-wave image can be used alone to determine the posture of the occluded patient, and in another embodiment, the millimeter-wave images acquired before and after occluding of the patient can be used together with the visible image acquired before occluding to determine the posture of the occluded patient.
[0093] Continuing with the detailed embodiment shown in Figure 5, for example, an RGB visible camera mounted on the ceiling of the room where the medical imaging unit is located acquires a high-resolution visible image of the patient on a table ready to be placed inside the medical imaging unit for image scanning. The millimeter-wave sensor system is also located on the ceiling of the room and may have two or more sensors to increase resolution, or it may have a moving sensor to increase resolution. However, only one millimeter-wave sensor in a fixed position may be used. The millimeter-wave sensor may be, for example, a multi-antenna array used to obtain high spatial resolution millimeter-wave images and may be a compact single-chip design operating in the 50-80 GHz band. This type of sensor can capture reflected signals of the scene and provide a 3D spatial signal. Importantly, millimeter-wave frequencies penetrate low dielectric optical materials such as wood, plastic, and textiles / fabrics. Simultaneously with the RGB camera acquiring the patient's visible image, the millimeter-wave sensor acquires the patient's millimeter-wave image.
[0094] As shown in Figure 2, a dedicated process is applied to each frame or image pair obtained from the two cameras. In the first step, a posture detection neural network is used to locate the patient's anatomical landmarks. If the patient is not occluded, all of the patient's landmarks are detectable by the network (upper panel of Figure 2). The millimeter-wave data acquired at this point is stored along with the detected landmarks to be used in later frames.
[0095] Once the occlusion is placed over the patient (the blanket in the lower part of Figure 2), the neural network cannot detect all anatomical landmarks. In this case, the millimeter-wave data of the current frame is compared with the stored millimeter-wave data of an available unoccluded frame. If the millimeter-wave data are similar (up to a predetermined threshold), patient movement between the two frames can be ruled out, and the stored coordinates of the occluded landmarks are transferred to the current frame. If the millimeter-wave data between the two frames shows a significant deviation, the operator is informed that patient movement has occurred after the placement of the occlusion. It should be noted that an appropriate threshold for the similarity of the millimeter-wave data can be easily calibrated in small volunteer studies using a specified motion profile, allowing observation of typical millimeter-wave data fluctuations with and without motion.
[0096] Therefore, the posture detection neural network is used to locate anatomical landmarks in the RGB data while the patient is unoccluded. Both these landmarks and their corresponding millimeter-wave sensor data are stored. When occlusion is placed on the patient, the current millimeter-wave sensor data is compared with the stored data from the unoccluded frame. If no change is detected, the stored landmarks are transferred to the current frame.
[0097] A posture detection neural network can be trained using a series of patient image pairs, with and without occlusion, with varying degrees of occlusion, and with and without motion between image pairs. This allows the neural network to analyze RGB / millimeter wave image pairs when occluded, then with RGB / millimeter wave image pairs when occluded, and determine the patient's posture even if the patient moves. Therefore, if the patient moves but that movement does not cause problems with subsequent medical image scans, the operator does not need to be informed about returning the patient to their original position, but can be informed that the patient has moved, in which case the operator can ask the patient to stop moving or simply temporarily remove the occlusion.
[0098] Millimeter-wave image data can be used not only to determine the patient's posture but also to detect metal objects on the patient's support structure. Due to its ability to penetrate clothing and other fabrics, even hidden objects such as bracelets and perforations can be detected. Neural networks can be used to assist in metal object detection, though they are not essential. The location of the detected metal object can then be visualized on the RGB image to provide guidance to the operator, who can then remove the object. This can be particularly important in MRI scans where ferromagnetic objects pose a significant safety concern.
[0099] Instead of using an RGB camera, an infrared or depth (time-of-flight) sensor can be used in conjunction with a neural network, which is trained to perform attitude detection on this specific image in combination with millimeter-wave image data.
[0100] By adding a millimeter-wave source to the system, the scene can be illuminated (active imaging), thereby enabling a higher signal-to-noise ratio (SNR) in the received data. However, if millimeter-wave radiation is not required, passive millimeter-wave imaging can be used.
[0101] In another exemplary embodiment, a computer program is provided, which is configured to perform any method step of the method according to one of the embodiments described above on a suitable apparatus or system.
[0102] Accordingly, the computer program can be stored in a computer unit which may be part of this embodiment. This computing unit can be configured to perform or trigger the steps of the method described above. Furthermore, it may be configured to operate the components of the system described above. The computing unit can be configured to operate automatically and / or to execute user instructions. The computer program can be loaded into the working memory of a data processor. Accordingly, the data processor can be configured to perform the method according to one of the embodiments described above.
[0103] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the outset and computer programs that modify an existing program to use the present invention through updates.
[0104] Furthermore, a computer program can provide all the steps necessary to satisfy the procedure of an exemplary embodiment of the method described above.
[0105] According to a further exemplary embodiment of the present invention, a computer-readable medium such as a CD-ROM or USB stick is presented, having a computer program stored thereon, which is described in the preceding section.
[0106] Computer programs may be stored and / or distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but they may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.
[0107] However, computer programs can also be presented through networks such as the World Wide Web and downloaded from such networks into the working memory of a data processor. According to a further exemplary embodiment of the present invention, a medium for making computer program elements available for download is provided, and the computer program is configured to perform the method according to one of the aforementioned embodiments of the present invention.
[0108] It should be noted that embodiments of the present invention are described with reference to different subject matter. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will understand from the above and below descriptions that, unless otherwise noted, any combination of features belonging to one type of subject matter, as well as any combination of features relating to different subject matter, are disclosed in this application. However, all features can be combined to provide a greater synergistic effect than the simple sum of the features.
[0109] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be considered illustrative or descriptive and not limiting. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and achieved by those skilled in the art in carrying out the claimed invention, from the examination of the drawings, disclosure and dependent claims.
[0110] In the claims, the words “comprising” do not exclude other components or steps, and the indefinite articles “a” or “an” do not exclude plurality. A single processor or other unit may perform the functions of several items mentioned in the claims. The mere fact that certain means are mentioned in different dependent claims does not imply that combinations of these means cannot be used advantageously. No reference numeral in the claims should be construed as limiting its scope.
Claims
1. A patient posture detection device, At least one millimeter-wave sensor and / or terahertz sensor, Processing unit and It has, The at least one millimeter-wave sensor and / or terahertz sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The at least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The at least one millimeter-wave sensor and / or terahertz sensor is configured to provide the processing unit with millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the posture of the patient on the patient support of the medical imaging unit, and the determination of the patient's posture on the patient support includes the use of the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit. The apparatus has a visible or infrared sensor, the visible or infrared sensor is configured to be located near the patient on the patient support of the medical imaging unit, the visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires medical image data of the patient, the visible or infrared sensor is configured to provide the visible or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit, and the determination of the patient's posture on the patient support includes the use of the visible or infrared image data of the patient on the patient support of the medical imaging unit. The apparatus wherein the millimeter-wave and / or terahertz image data of a patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame after the first time frame, and the visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
2. The apparatus according to claim 1, wherein the at least one millimeter-wave sensor and / or terahertz sensor is configured to move to a plurality of positions to acquire the millimeter-wave image data of the patient on the patient support of the medical imaging unit.
3. The apparatus according to claim 1 or 2, wherein the at least one millimeter-wave sensor and / or terahertz sensor comprises a plurality of millimeter-wave sensors and / or terahertz sensors spaced apart from each other.
4. The apparatus according to claim 1, wherein the processing unit is configured to determine the locations of a plurality of landmarks of the patient in the first time frame, the determination of the locations of the plurality of landmarks of the patient in the first time frame includes the use of a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible or infrared image of the patient on the patient support of the medical imaging unit; the processing unit is configured to determine the locations of a plurality of landmarks of the patient in the second time frame, the determination of the locations of the plurality of landmarks of the patient in the second time frame includes the determined locations of the plurality of landmarks of the patient in the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit; and the determination of the posture of the patient on the patient support includes the use of the determined locations of the plurality of landmarks of the patient in the second time frame.
5. The apparatus according to any one of claims 1 to 4, wherein the visible or infrared image data of the patient on the patient support of the medical imaging unit is acquired simultaneously with the second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit, and has a second visible or infrared image of the patient on the patient support of the medical imaging unit.
6. The apparatus according to claim 5, wherein the determination of the positions of the plurality of landmarks of the patient in the second time frame includes the use of the second visible or infrared image of the patient on the patient support of the medical imaging unit.
7. The apparatus according to any one of claims 4 to 6, wherein the determination of the patient's posture on the patient support includes the use of a trained neural network.
8. The apparatus according to claim 7, wherein the trained neural network is trained on a plurality of image pairs of one or more reference persons, the plurality of image pairs comprising millimeter-wave and / or terahertz images of the one or more reference persons and associated visible or infrared images of the one or more persons acquired simultaneously with the millimeter-wave and / or terahertz images of the one or more reference persons.
9. The apparatus according to claim 8, wherein the plurality of image pairs of the one or more reference persons include image data of the one or more reference persons having different degrees of body occlusion relative to visible image data or infrared image data.
10. A patient posture detection system, Medical imaging unit, At least one millimeter-wave sensor and / or terahertz sensor, Processing unit and It has, The at least one millimeter-wave sensor and / or terahertz sensor is positioned near the patient on the patient support of the medical imaging unit. The at least one millimeter-wave sensor and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the patient's medical image data. The at least one millimeter-wave sensor and / or terahertz sensor is configured to provide the processing unit with millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit, the processing unit is configured to determine the posture of the patient on the patient support of the medical imaging unit, the determination of the posture of the patient on the patient support includes the use of the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit, the system further includes a visible or infrared sensor, the visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit, the visible or infrared sensor is configured to acquire visible or infrared image data of the patient on the patient support of the medical imaging unit before the medical image data of the patient is acquired by the medical imaging unit, the visible or infrared sensor is configured to provide the processing unit with the visible or infrared image data of the patient, the determination of the posture of the patient on the patient support includes the use of the visible or infrared image data of the patient on the patient support of the medical imaging unit, The millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support stand of the medical imaging unit acquired in a second time frame after the first time frame, and the visible or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible or infrared image of the patient on the patient support of the medical imaging unit acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
11. A method for detecting patient posture, Before the medical imaging unit acquires the patient's medical image data, the medical imaging unit acquires millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit using at least one millimeter-wave sensor and / or terahertz sensor located near the patient on the patient support of the medical imaging unit. The steps include providing a processing unit with millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit using at least one millimeter-wave sensor and / or terahertz sensor, The processing unit performs the steps of determining the posture of the patient on the patient support on the medical imaging unit, It has, The determination of the patient's posture on the patient support comprises the use of millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit, wherein a visible sensor or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit, the visible sensor or infrared sensor is configured to acquire visible image data or infrared image data of the patient on the patient support of the medical imaging unit before the medical image data of the patient is acquired by the medical imaging unit, and the visible sensor or infrared sensor is configured to provide the visible image data or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit. The method for determining the posture of the patient on the patient support comprises using visible image data or infrared image data of the patient on the patient support of the medical imaging unit, wherein the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit comprises a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired in a second time frame after the first time frame, and the visible or infrared image data of the patient on the patient support of the medical imaging unit comprises a first visible or infrared image of the patient on the patient support of the medical imaging unit acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.
12. A computer program for controlling the apparatus according to any one of claims 1 to 9 or the system according to claim 10, configured to perform the method according to claim 11 when executed by a processor.
13. A computer-readable medium storing the computer program described in claim 12.