Patient posture detection device

Abstract

The present invention relates to a patient posture detection device (10) comprising: at least one millimeter wave and / or terahertz sensor (20); and a processing unit (30); wherein the at least one millimeter wave and / or terahertz sensor is configured to be positioned near a patient on a patient support of a medical imaging unit; wherein the at least one millimeter wave and / or terahertz sensor is configured to: acquire millimeter wave and / or terahertz image data of the patient prior to acquiring, by the medical imaging unit, medical image data of the patient on the patient support of the medical imaging unit; wherein the at least one millimeter wave and / or terahertz sensor is configured to provide millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit; and wherein the processing unit is configured to determine a posture of the patient on the patient support of the medical imaging unit, determining the posture of the patient on the patient support includes utilizing millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

Description

Technical Field

[0001] The present invention relates to a patient pose detection device, a patient pose detection system, a patient pose detection method, a computer program unit, and a computer-readable medium. Background Art

[0002] Preparing a patient for a medical imaging scan, examination, or test is a time-consuming and error-prone task. To simplify and accelerate the clinical workflow, using an optical sensor (camera) to automate this part of the examination is a viable option. One of the key requirements of such a system is to highly accurately locate the anatomical key points or landmarks of the patient, which are used, for example, to automatically determine the scan position, the pose of the patient (feet first / head first, supine / prone, etc.), or for safety functions such as collision detection.

[0003] When relying on an optical sensor, occlusion of the patient can complicate the localization of anatomical key points or landmarks. Figure 1 An example of such a situation is shown, where a blanket is placed on the patient, making it impossible to accurately locate the key points or landmarks of the lower limbs. It is necessary to solve these problems. US2019 / 0143145A1 discloses systems and methods for patient monitoring for radiotherapy. US2020 / 281539A1 discloses the creation of a digital twin for medical examinations. US2007 / 014391A1 discloses systems and methods for treating a patient using radiation. Summary of the Invention

[0004] It would be advantageous to have an improved technique to assist in acquiring the pose of a patient prior to a medical image scan. The object of the present invention is solved by the subject matter of the independent claims, wherein further embodiments are incorporated into the dependent claims.

[0005] In a first aspect, there is provided a patient pose detection device, comprising:

[0006] at least one millimeter-wave and / or terahertz sensor; and

[0007] a processing unit.

[0008] The at least one millimeter-wave and / or terahertz sensor is configured to be positioned near a patient on a patient support of a medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to acquire millimeter-wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. The processing unit is configured to determine the pose of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0009] In this way, millimeter-wave radiation and / or terahertz radiation can be propagated through various materials such that the pose of the patient on a patient support (such as a bed) can be determined even if the patient is occluded (such as partially covered by a blanket). This can ensure that the patient can be correctly positioned within the medical imaging unit (such as a CT scanner, an MRI scanner, a PET scanner, etc.) before being placed inside the medical imaging unit for a medical imaging scan or examination.

[0010] It is noted that either millimeter-wave sensing in the range of 30 - 300 GHz or terahertz sensing in the range of 300 - 3000 GHz can be used alone, as both modalities enable the acquisition of images through occlusions. However, by using both a millimeter-wave sensor and a terahertz sensor simultaneously, the bandwidth of the images can be effectively increased.

[0011] In an example, the at least one millimeter-wave and / or terahertz sensor is configured to move to multiple positions to acquire the millimeter-wave image data of the patient on the patient support of the medical imaging unit.

[0012] In this way, by moving one or more millimeter-wave and / or terahertz sensors, a higher-resolution millimeter-wave image of the patient can be obtained, thereby improving the ability to accurately determine the pose of the patient.

[0013] In an example, the at least one millimeter-wave and / or terahertz sensor includes a plurality of millimeter-wave and / or terahertz sensors spaced apart from each other.

[0014] By using a plurality of millimeter-wave and / or terahertz sensors to view the patient from different positions (and thus from different angles), a higher-resolution millimeter-wave and / or terahertz image of the patient can be obtained, thereby improving the ability to accurately determine the pose of the patient.

[0015] In an example, the device includes a visible light or infrared sensor. The visible light or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to acquire visible light or infrared image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit. Determining the pose of the patient on the patient support may include using the visible light or infrared image data of the patient on the patient support of the medical imaging unit.

[0016] In this way, high-resolution visible light or infrared images can be used to enhance millimeter-wave and / or terahertz imaging, facilitating better determination of the patient's pose.

[0017] In an example, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame. The visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, the first visible light or infrared image being acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0018] Thus, a first millimeter-wave image and a second millimeter-wave image, or a first terahertz image and a second terahertz image, can be acquired. When a millimeter-wave sensor and a terahertz sensor are used simultaneously, they will acquire simultaneous images, and the simultaneous images can be fused in effect to form a first millimeter-wave / terahertz image and fused again at a later acquisition time to form a second millimeter-wave / terahertz image.

[0019] In this way, a first millimeter wave and / or terahertz image and an associated visible image can be acquired before the patient is partially occluded (e.g., a blanket is placed over a part of their body). Then, the unoccluded visible light or infrared image can be used together with the associated millimeter wave and / or terahertz image to provide information on how to better interpret the millimeter wave and / or terahertz image with respect to the patient's pose. Then the patient can be at least partially occluded, e.g., a blanket is placed over a part of their body. Subsequently, a millimeter wave and / or terahertz image that can penetrate the occluder (penetrate the blanket) is acquired. Then the information derived from the unoccluded visible light or infrared image and the associated millimeter wave and / or terahertz image can be used to better interpret the millimeter wave and / or terahertz image of the occluded patient, enabling the accurate determination of the patient's pose.

[0020] In an example, the processing unit is configured to determine the positions of a plurality of landmarks of the patient at a first time frame. Determining the positions of the plurality of landmarks of the patient at the first time frame includes using the first millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit and the first visible light or infrared image of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the positions of the plurality of landmarks of the patient at a second time frame. Determining the positions of the plurality of landmarks of the patient at the second time frame includes using the positions of the plurality of landmarks of the patient determined at the first time frame and the second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support may include using the positions of the plurality of landmarks of the patient determined at the second time frame.

[0021] In other words, the first millimeter wave and / or terahertz image and the associated visible light or infrared image are used to determine the positions of landmark features of the patient such as the ankles, knees, hips, wrists, elbows, shoulders, and head, where the high-resolution visible light / infrared image helps to localize the landmark features within the millimeter wave and / or terahertz image. Then, in effect, the combined millimeter wave (terahertz) / visible light or infrared image and its landmarks can be used to help localize the same landmarks in a subsequent millimeter wave image of the patient (where the patient is now partially occluded). This facilitates the accurate determination of the pose of the patient when partially occluded.

[0022] In an example, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes the second visible light or infrared image of the patient on the patient support of the medical imaging unit, and the second visible light or infrared image is acquired simultaneously with the second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0023] In an example, determining the positions of a plurality of landmarks of a patient at a second time frame includes using the second visible light or infrared image of the patient on the patient support of the medical imaging unit.

[0024] In this way, visible light / infrared and millimeter wave and / or terahertz image data are used in combination before and after the patient is partially occluded (e.g., a part of the body is covered with a blanket) so as to enable determination of the pose of the patient when the patient is partially occluded.

[0025] In an example, determining the pose of a patient on a patient support includes using a trained neural network.

[0026] In an example, the trained neural network has been trained based on a plurality of image pairs of one or more reference persons. The plurality of image pairs include millimeter wave and / or terahertz images of the one or more reference persons and associated visible light or infrared images of the one or more persons acquired simultaneously with the millimeter wave and / or terahertz images of the one or more reference persons.

[0027] In an example, the plurality of image pairs of the one or more reference persons include image data of the one or more reference persons with different degrees of body occlusion with respect to visible light or infrared image data.

[0028] Therefore, both millimeter wave and visible light / infrared images of persons who are unoccluded in two images of a first image set and are occluded to different degrees in two images of another image set are used to train the neural network. This enables the neural network to receive millimeter wave and visible / infrared image pairs of an unoccluded patient and receive subsequent millimeter wave and visible / infrared image pairs of an occluded patient, from which the pose of the occluded patient can be accurately determined.

[0029] In a second aspect, there is provided a patient pose detection system, comprising:

[0030] A medical imaging unit;

[0031] At least one millimeter wave and / or terahertz sensor; and

[0032] A processing unit.

[0033] The at least one millimeter wave and / or terahertz sensor is positioned near a patient on a patient support of a medical imaging unit. The at least one millimeter wave and / or terahertz sensor is configured to acquire millimeter wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The at least one millimeter wave and / or terahertz sensor is configured to provide the millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. The processing unit is configured to determine the pose of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support includes using the millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0034] In a third aspect, there is provided a method for detecting a patient's pose, comprising:

[0035] Before a medical imaging unit acquires medical image data of a patient, at least one millimeter wave and / or terahertz sensor positioned near the patient on a patient support of the medical imaging unit acquires millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit;

[0036] The at least one millimeter wave and / or terahertz sensor provides the millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. And

[0037] The processing unit determines the pose of the patient on the patient support of the medical imaging unit, wherein determining the pose of the patient on the patient support includes using the millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0038] In one aspect, there is provided a computer program unit for controlling the apparatus according to the first aspect, which when run by a processor is configured to execute the method according to the third aspect.

[0039] In one aspect, there is provided a computer program unit for controlling the system according to the second aspect, which when run by a processor is configured to execute the method according to the third aspect.

[0040] Thus, according to various aspects, there is provided a computer program unit for controlling one or more of the apparatuses / systems as described above, which when run by a processor is adapted to execute the methods as described above.

[0041] According to another aspect, there is provided a computer-readable medium storing a computer unit as described above.

[0042] The computer program unit may be, for example, a software program, but may also be an FPGA, a PLD, or any other suitable digital device.

[0043] Advantageously, the benefits provided by any of the above aspects apply equally to all other aspects, and vice versa.

[0044] The above aspects and examples will become apparent and be elucidated in accordance with the embodiments described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Exemplary embodiments will be described below with reference to the drawings:

[0046] Figure 1 A patient preparation scenario before imaging using a medical imaging unit is shown, where the patient is partially occluded due to a blanket, making it extremely difficult to position the lower limbs;

[0047] Figure 2 An example of a patient pose detection device is shown;

[0048] Figure 3 An example of a patient pose detection system is shown;

[0049] Figure 4 A patient pose detection method is shown; and

[0050] Figure 5 A workflow of a detailed embodiment of a patient pose detection device, system, and method is shown. DETAILED DESCRIPTION

[0051] Figure 2An example of a patient pose detection device 10 is shown, which includes at least one millimeter-wave and / or terahertz sensor 20 and a processing unit 30. The at least one millimeter-wave and / or terahertz sensor is configured to be positioned near a patient on a patient support of a medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to: acquire millimeter-wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. The processing unit is configured to determine the pose of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0052] In this way, millimeter-wave radiation and / or terahertz radiation can be propagated through various materials so that the pose of the patient on a patient support (such as a bed) can be determined even if the patient is occluded (such as partially covered by a blanket). This can ensure that the patient can be correctly positioned within the medical imaging unit (such as a CT scanner, an MRI scanner, a PET scanner, etc.) before being placed inside the medical imaging unit for a medical imaging scan or examination.

[0053] It is worth noting that either only millimeter-wave sensing in the range of 30 - 300 GHz or only terahertz sensing in the range of 300 - 3000 GHz can be used, because both modalities enable the acquisition of images through occlusions. However, by using a millimeter-wave sensor and a terahertz sensor simultaneously, the bandwidth of the image can be effectively increased.

[0054] In an example, the processing unit is configured to control at least one millimeter-wave and / or terahertz sensor to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the medical image data of the patient.

[0055] According to an example, the at least one millimeter-wave and / or terahertz sensor is configured to move to multiple positions to acquire millimeter-wave image data of the patient on the patient support of the medical imaging unit.

[0056] In this way, by moving one or more millimeter-wave and / or terahertz sensors, a higher-resolution millimeter-wave image of the patient can be obtained, thereby improving the ability to accurately determine the patient's pose.

[0057] According to an example, the at least one millimeter-wave and / or terahertz sensor includes a plurality of millimeter-wave and / or terahertz sensors spaced apart from each other.

[0058] By using a plurality of millimeter-wave and / or terahertz sensors to view the patient from different positions (and thus from different angles), a higher-resolution millimeter-wave and / or terahertz image of the patient can be obtained, thereby improving the ability to accurately determine the patient's pose.

[0059] According to an example, the device includes a visible light or infrared sensor. The visible light or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to acquire visible light or infrared image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit. Determining the pose of the patient on the patient support may include using the visible light or infrared image data of the patient on the patient support of the medical imaging unit.

[0060] In this way, high-resolution visible light or infrared images can be used to enhance millimeter-wave and / or terahertz imaging, facilitating better determination of the patient's pose.

[0061] In an example, the processing unit is configured to: control the visible light or infrared sensor to acquire visible light or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires medical image data of the patient.

[0062] According to an example, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame. The visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, the first visible light or infrared image being acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0063] Thus, a first millimeter wave image and a second millimeter wave image can be acquired, or a first terahertz image and a second terahertz image can be acquired. When a millimeter wave sensor and a terahertz sensor are used simultaneously, they will acquire simultaneous images, and the simultaneous images can be fused in effect to form a first millimeter wave / terahertz image and fused again at a later acquisition time to form a second millimeter wave / terahertz image.

[0064] In this way, a first millimeter wave and / or terahertz image and a related visible image of the patient can be acquired before the patient is partially occluded (e.g., a blanket is placed over a part of their body). Then the unoccluded visible light or infrared image can be used together with the associated millimeter wave and / or terahertz image to provide information on how to better interpret the millimeter wave and / or terahertz image with respect to the patient's pose. Then the patient can be at least partially occluded, e.g., a blanket is placed over a part of their body. Subsequently, millimeter wave and / or terahertz images that can see through the occluder (see through the blanket) are acquired. Then the information derived from the unoccluded visible light or infrared image and the associated millimeter wave and / or terahertz image can be used to better interpret the millimeter wave and / or terahertz image of the occluded patient, enabling the accurate determination of the patient's pose.

[0065] According to an example, the processing unit is configured to determine the positions of a plurality of landmarks of the patient at a first time frame. Determining the positions of the plurality of landmarks of the patient at the first time frame includes using the first millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit and the first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the processing unit is configured to determine the positions of the plurality of landmarks of the patient at a second time frame. Determining the positions of the plurality of landmarks of the patient at the second time frame includes using the positions of the plurality of landmarks of the patient determined at the first time frame and the second millimeter and / or terahertz wave image of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support can include using the positions of the plurality of landmarks of the patient determined at the second time frame.

[0066] In other words, the first millimeter wave and / or terahertz image and the associated visible light or infrared image are used to determine the positions of landmark features of a patient such as the ankles, knees, hips, wrists, elbows, shoulders, and head, wherein the high-resolution visible light / infrared image helps to localize the landmark features within the millimeter wave and / or terahertz image. Then, in effect, the combined millimeter wave (terahertz) / visible light or infrared image and its landmarks can be used to assist in localizing the same landmarks in a subsequent millimeter wave image of the patient (where the patient is now partially occluded). This facilitates the accurate determination of the pose of the patient when partially occluded.

[0067] According to an example, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a second visible light or infrared image of the patient on the patient support of the medical imaging unit, and the second visible light or infrared image is acquired simultaneously with a second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0068] According to an example, determining the positions of multiple landmarks of a patient at a second time frame includes using the second visible light or infrared image of the patient on the patient support of the medical imaging unit.

[0069] In this way, visible light / infrared and millimeter wave and / or terahertz image data are used in combination before and after the patient is partially occluded (e.g., with a blanket covering a part of their body) so as to enable the determination of the pose of the patient when partially occluded.

[0070] According to an example, determining the pose of the patient on the patient support includes using a trained neural network.

[0071] According to an example, the trained neural network is trained based on multiple image pairs of one or more reference persons. The multiple image pairs include millimeter wave and / or terahertz images of the one or more reference persons and the associated visible light or infrared images of the one or more persons acquired simultaneously with the millimeter wave and / or terahertz images of the one or more reference persons.

[0072] In an example, the multiple image pairs are reference images of a person on the patient support. Thus, images of persons not on the patient support can be used to train the neural network, and in fact, it can have a completely different scenario, and the neural network will perform well for the patient support scenario. However, the reference image pairs can include image pairs of multiple persons on the patient support.

[0073] According to an example, the multiple image pairs of one or more reference persons include image data of the one or more reference persons having different degrees of body occlusion with respect to the visible light or infrared image data.

[0074] Thus, both millimeter-wave and visible / infrared images of a person who is unoccluded in two images of a first image set and is occluded to varying degrees in two images of an additional image set are used to train a neural network. This enables the neural network to receive millimeter-wave and visible / infrared image pairs of non-occluded patients and subsequent millimeter-wave and visible / infrared image pairs of occluded patients from which the pose of the occluded patient can be accurately determined.

[0075] Figure 3 An example of a patient pose detection system 100 is shown, which includes a medical imaging unit 110, at least one millimeter-wave and / or terahertz sensor 20, and a processing unit 30. The at least one millimeter-wave and / or terahertz sensor is positioned near a patient on a patient support of the medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to: acquire millimeter-wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The at least one millimeter-wave and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit. The processing unit is configured to determine the pose of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support includes utilizing the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0076] In an example, the processing unit is configured to control the at least one millimeter-wave and / or terahertz sensor to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires the medical image data of the patient.

[0077] In an example, the at least one millimeter-wave and / or terahertz sensor is configured to move to multiple positions to acquire millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0078] In an example, the at least one millimeter-wave and / or terahertz sensor includes a plurality of millimeter-wave and / or terahertz sensors spaced apart from each other.

[0079] In an example, the system includes a visible light or infrared sensor. The visible light or infrared sensor is positioned near the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to acquire visible light or infrared image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit. Determining the pose of the patient on the patient support may include using the visible light or infrared image data of the patient on the patient support of the medical imaging unit.

[0080] In an example, the processing unit is configured to: control the visible light or infrared sensor to acquire visible light or infrared image data of the patient on the patient support of the medical imaging unit before the medical imaging unit acquires medical image data of the patient.

[0081] In an example, the millimeter wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame. The visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the first visible light or infrared image is acquired simultaneously with the first millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0082] In an example, the processing unit is configured to determine the positions of a plurality of landmarks of a patient at a first time frame. Determining the positions of the plurality of landmarks of the patient at the first time frame includes using a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and a first visible light or infrared image of the patient on the patient support of the medical imaging unit. The processing unit is configured to determine the positions of the plurality of landmarks of the patient at a second time frame, wherein determining the positions of the plurality of landmarks of the patient at the second time frame includes using the determined positions of the plurality of landmarks of the patient at the first time frame and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support includes using the positions of the plurality of landmarks of the patient determined at the second time frame.

[0083] In an example, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a second visible light or infrared image of the patient on the patient support of the medical imaging unit, and the second visible light or infrared image is acquired simultaneously with a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0084] In an example, determining the positions of the plurality of landmarks of the patient at the second time frame includes using the second visible light or infrared image of the patient on the patient support of the medical imaging unit.

[0085] In an example, determining the pose of the patient on the patient support includes using a trained neural network.

[0086] In an example, the trained neural network has been trained based on a plurality of image pairs of one or more reference persons. The plurality of image pairs includes millimeter-wave and / or terahertz images of one or more reference persons and associated visible light or infrared images of one or more reference persons acquired simultaneously with the millimeter-wave and / or terahertz images of one or more reference persons.

[0087] In an example, the plurality of image pairs of one or more reference persons includes image data of one or more reference persons having different degrees of body occlusion with respect to visible light or infrared image data.

[0088] Figure 4 A patient pose detection method 200 is shown, including:

[0089] Before the medical imaging unit acquires medical image data of a patient, at least one millimeter-wave and / or terahertz sensor near the patient positioned on the patient support of the medical imaging unit acquires millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit;

[0090] The at least one millimeter-wave and / or terahertz sensor provides the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit; and

[0091] The processing unit determines the pose of the patient on the patient support of the medical imaging unit, wherein determining the pose of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit.

[0092] In an example, the method includes the processing unit controlling at least one millimeter-wave and / or terahertz sensor to acquire millimeter-wave and / or terahertz image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires medical image data of the patient.

[0093] In an example, the method includes moving the at least one millimeter-wave and / or terahertz sensor to multiple positions to acquire millimeter-wave and / or terahertz image data of a patient on a patient support of a medical imaging unit.

[0094] In an example, the at least one millimeter-wave and / or terahertz sensor includes a plurality of millimeter-wave and / or terahertz sensors spaced apart from each other.

[0095] In an example, the system includes a visible light or infrared sensor, wherein the visible light or infrared sensor is positioned near the patient on the patient support of the medical imaging unit. The method includes, before the medical imaging unit acquires medical image data of a patient, the visible light or infrared sensor acquiring visible light or infrared image data of the patient on the patient support of the medical imaging unit. The method includes providing the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit through the visible light or infrared sensor. Determining the pose of the patient on the patient support may include using the visible light or infrared image data of the patient on the patient support of the medical imaging unit.

[0096] In an example, the method includes the processing unit controlling the visible light or infrared sensor to acquire visible light or infrared image data of a patient on a patient support of a medical imaging unit before the medical imaging unit acquires medical image data of the patient.

[0097] In an example, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame. The visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the first visible light or infrared image is acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0098] In an example, the method includes determining, by a processing unit, the positions of a plurality of landmarks of a patient at a first time frame. Determining the positions of the plurality of landmarks of the patient at the first time frame includes using the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit and the first visible light or infrared image of the patient on the patient support of the medical imaging unit. The method includes determining, by the processing unit, the positions of a plurality of landmarks of the patient at a second time frame. Determining the positions of the plurality of landmarks of the patient at the second time frame includes using the determined positions of the plurality of landmarks of the patient at the first time frame and the second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit. Determining the pose of the patient on the patient support may include using the determined positions of the plurality of landmarks of the patient at the second time frame.

[0099] In an example, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a second visible light or infrared image of the patient on the patient support of the medical imaging unit, and the second visible light or infrared image is acquired simultaneously with the second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

[0100] In an example, determining the positions of a plurality of landmarks of the patient at a second time frame includes using the second visible light or infrared image of the patient on the patient support of the medical imaging unit.

[0101] In an example, determining the pose of the patient on the patient support includes using a trained neural network.

[0102] In an example, the trained neural network has been trained based on multiple image pairs of one or more reference persons. The multiple image pairs include millimeter wave and / or terahertz images of one or more reference persons and associated visible light or infrared images of one or more reference persons acquired simultaneously with the millimeter wave and / or terahertz images of one or more reference persons.

[0103] In an example, the multiple image pairs of one or more reference persons include image data of one or more reference persons having different degrees of body occlusion with respect to visible light or infrared image data.

[0104] Reference will now be made to Figure 5 a patient pose detection device, system, and method described in more detail, where the focus of the discussion is on millimeter wave radiation, but it is also applicable to terahertz radiation.

[0105] The inventors have realized that millimeter-wave radiation (sometimes referred to as millimeter radio-frequency wave radiation) can be utilized in a completely new way to determine the pose of a patient who is about to undergo a medical imaging scan or an examination / test. Millimeter-wave sensors or scanners have been developed for detecting sealed objects, such as for airport security. See, for example: D. McMakin et al., “New Improvements to Millimeter-Wave Body Scanners”, Proceedings of 3DBODY.TECH 2017, 8th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Montreal, Canada, October 11–12, 2017, pp. 263–271; D. Li et al., “Multi-Person Action Recognition in Microwave Sensors”, Oral Session H1: Emerging Multimedia Applications, MM'20, October 12–16, 2020, Seattle, WA, USA, pp. 411–420; R. Feger et al., “A 77-GHz FMCW MIMO Radar Based on an SiGe Single-Chip Transceiver”, IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, VOL. 57, NO. 5, May 2009, pp. 1020–1035; F. Garcia-Rial et al., “Combining Commercially Available Active and Passive Sensors Into a Millimeter-Wave Imager for Concealed Weapon Detection”, IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, VOL. 67, NO. 3, March 2019, pp. 1167–1183; Y.Lu et al., "Study on feasibility of remote metal detection using millimeter wave radar for convenient and efficient security check", CCF Transactions on Pervasive Computing and Interaction (2021) 3, pp. 284-299; G. Tzydynzhapov et al., "New Real-Time Sub-Terahertz Security Body Scanner", Journal of Infrared, Millimeter, and Terahertz Waves, 41, 2020, pp. 632-641; R. Z. Syeda et al., "Sparse MIMO Array for Improved 3D mm-Wave Imaging Radar", Proceedings of the 17th European Radar Conference, 2021, pp. 342-345.

[0106] Figure 5 Shows a workflow related to detailed embodiments of the apparatus, system, and method that utilizes a combination of millimeter wave images and visible light images, acquired before and after partial occlusion of a patient to determine the pose of the occluded patient. However, in one embodiment, millimeter wave images can be used alone to determine the pose of the patient after occlusion, while in another embodiment, millimeter wave images acquired before and after patient occlusion and visible images acquired before occlusion can be used together to determine the pose of the patient after occlusion.

[0107] Continue Figure 5In the detailed embodiment shown, for example, an RGB visible light camera installed on the ceiling of the room where the medical image acquisition unit is located acquires high-resolution visible light images of a patient who is about to be placed on a table for image scanning within the medical image acquisition unit. The millimeter-wave sensor system is also located on the ceiling of the room and may have multiple sensors to provide higher resolution, or may have one or more movable sensors to also provide higher resolution. However, only one millimeter-wave sensor located at a fixed position can be used. The millimeter-wave sensor can be a multi-antenna array for acquiring millimeter-wave images with high spatial resolution, or can be a compact single-chip design operating in the 50 - 80 GHz frequency band. Such sensors capture the reflected signals of the scene and can provide 3D spatial signals. Importantly, millimeter-wave frequencies can penetrate low-dielectric optical materials such as wood, plastic, and textiles / fabrics. The millimeter-wave sensor acquires millimeter-wave images of the patient while the RGB camera acquires visible images of the patient.

[0108] For each frame or pair of images obtained from the two cameras, specialized processing is applied, as Figure 2 shown. In the first step, a pose detection neural network is used to locate the anatomical landmarks of the patient. If the patient is not occluded, the network can detect all the landmarks of the patient ( Figure 2 top row in). The millimeter-wave data for this time point is stored together with the detected landmarks for use in subsequent frames.

[0109] Once an occluder is placed on the patient ( Figure 2 blanket in the bottom row), the neural network cannot detect all the anatomical landmarks. In this case, the millimeter-wave data of the current frame is compared with the stored millimeter-wave data of the available unoccluded frames. If the millimeter-wave data is similar (reaching a predefined threshold), the patient movement between the two frames can be excluded, and the stored coordinates of the occluded landmarks are transferred to the current frame. If the millimeter-wave data between the two frames shows a significant deviation, the operator is notified that patient movement has occurred after the placement of the occluder. It is worth noting that a suitable threshold for millimeter-wave data similarity can be easily calibrated in a small volunteer study by guiding the movement curve, thus allowing the observation of typical millimeter-wave data changes in cases of movement and no movement.

[0110] Therefore, when the patient is not occluded, a pose detection neural network is used to locate the anatomical landmarks in the RGB data. These landmarks and the corresponding millimeter-wave sensor data are stored. Once the patient is occluded, the current millimeter-wave sensor is compared with the stored data of the unoccluded frames. If no change is detected, the stored landmarks are transferred to the current frame.

[0111] The pose detection neural network can be trained on a series of image pairs of one or more patients, where the series of image pairs includes those without occlusion and with various degrees of occlusion, as well as those with no movement between image pairs and those with movement between image pairs, so that the neural network can analyze RGB / mmWave image pairs in non-occluded situations, then analyze RGB / mmWave image pairs in occluded situations, and determine the pose of the patient even if the patient has moved. Therefore, if the patient has moved, but the movement does not cause problems for subsequent medical image scans, the operator does not need to be informed about restoring the patient to the original position, but can be informed that the patient has moved. In this case, the operator can ask the patient to stop moving or just temporarily remove the occlusion.

[0112] In addition to being used to determine the pose of the patient, the mmWave image data can also be used to detect metal objects on the patient support. Since it can penetrate clothes and other fabrics, even hidden objects such as bracelets and body piercing ornaments can be detected. The neural network can be used to assist in metal object detection, but this is not necessary. Then the position of the detected metal object can be visualized on the RGB image to provide guidance to the operator, and then the operator can remove the object. This is especially important for MRI scans because ferromagnetic objects are a serious safety hazard.

[0113] In addition to using an RGB camera, an infrared or depth (time-of-flight) sensor can also be used, as well as a neural network trained to perform pose detection on this specific image in combination with the mmWave image data.

[0114] An mmWave source can be added to the system to illuminate the scene (active imaging), so that the received data has a higher signal-to-noise ratio. However, where a source of mmWave radiation is not required, passive mmWave imaging can be utilized.

[0115] In another exemplary embodiment, a computer program or computer program unit is provided, characterized in that it is configured to execute the method steps of any method according to one of the foregoing embodiments on a suitable device or system.

[0116] The computer program unit can thus be stored on a computing unit, which can also be part of the embodiment. The computing unit can be configured to execute the steps of the above method or cause the execution of the steps of the above method. In addition, it can be configured to operate the components of the above system. The computing unit can be configured to automatically operate and / or execute the commands of the user. The computer program can be loaded into the working memory of the data processor. The data processor can thus be equipped to execute the method according to one of the foregoing embodiments.

[0117] This exemplary embodiment of the invention covers both computer programs that use the invention from the start and computer programs that transform existing programs into programs that use the invention by means of updates.

[0118] In addition, the computer program unit may be capable of providing all the necessary steps to complete the process of the exemplary embodiment of the method as described above.

[0119] According to another exemplary embodiment of the invention, a computer-readable medium is proposed, such as a CD-ROM, a USB memory stick, etc., wherein the computer-readable medium has a computer program unit stored thereon, and the computer program unit is as described in the previous part.

[0120] The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium that is provided together with other hardware or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0121] However, the computer program may also be provided via a network similar to the World Wide Web, and the computer program can be downloaded from such a network into the working memory of a data processor. According to another exemplary embodiment of the invention, a medium for making a computer program unit available for download is provided, and the computer program unit is arranged to execute one of the previously described embodiments of the invention.

[0122] It must be noted that the embodiments of the invention are described with reference to different subjects. In particular, some embodiments are described with reference to method-type claims, while other embodiments are described with reference to device-type claims. However, as can be derived by those skilled in the art from the above and the following description, unless otherwise indicated, any combination between features related to different subjects is also considered to be disclosed by this application, in addition to any combination of specific ones belonging to the same type of subject. However, all features can be combined to provide a synergistic effect that exceeds the simple addition of the said features.

[0123] Although the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative or exemplary, rather than restrictive. The invention is not limited to the disclosed embodiments. Those skilled in the art can understand and implement other variations of the disclosed embodiments when practicing the claimed invention by studying the drawings, the disclosure, and the dependent claims.

[0124] In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used advantageously. Any reference signs in the claims shall not be construed as limiting the scope.

Claims

1. A patient posture detection device (10), comprising: At least one millimeter-wave and / or terahertz sensor (20); And A processing unit (30); Wherein, the at least one millimeter-wave and / or terahertz sensor is configured to be positioned near a patient on a patient support of a medical imaging unit; Wherein, the at least one millimeter-wave and / or terahertz sensor is configured to: acquire millimeter-wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit; Wherein, the at least one millimeter-wave and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit; and Wherein, the processing unit is configured to determine the pose of the patient on the patient support of the medical imaging unit, wherein determining the pose of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit; Wherein, the device includes a visible light or infrared sensor, wherein the visible light or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit, wherein the visible light or infrared sensor is configured to: acquire visible light or infrared image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit, wherein the visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit, and wherein determining the pose of the patient on the patient support includes using the visible light or infrared image data of the patient on the patient support of the medical imaging unit; wherein, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter-wave and / or / terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame, and wherein, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the first visible light or infrared image is acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

2. The device according to claim 1, wherein, The at least one millimeter-wave and / or terahertz sensor is configured to move to multiple positions to acquire the millimeter-wave image data of the patient on the patient support of the medical imaging unit.

3. The device according to any one of claims 1 - 2, wherein, The at least one millimeter wave and / or terahertz sensor includes a plurality of millimeter wave and / or terahertz sensors spaced apart from each other.

4. The device according to claim 1, wherein, The processing unit is configured to determine the positions of a plurality of landmarks of the patient at the first time frame, wherein determining the positions of the plurality of landmarks of the patient at the first time frame includes using the first millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit and the first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the processing unit is configured to determine the positions of the plurality of landmarks of the patient at the second time frame, wherein determining the positions of the plurality of landmarks of the patient at the second time frame includes using the determined positions of the plurality of landmarks of the patient at the first time frame and the second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit, and wherein determining the pose of the patient on the patient support includes using the determined positions of the plurality of landmarks of the patient at the second time frame.

5. The device according to any one of claims 1 - 4, wherein, The visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a second visible light or infrared image of the patient on the patient support of the medical imaging unit, the second visible light or infrared image being acquired simultaneously with the second millimeter wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

6. The device according to claim 5 when dependent on claim 1, wherein, Determining the positions of the plurality of landmarks of the patient at the second time frame includes using the second visible light or infrared image of the patient on the patient support of the medical imaging unit.

7. The device according to any one of claims 4 - 6, wherein, Determining the pose of the patient on the patient support includes using a trained neural network.

8. The device according to claim 7, wherein, The trained neural network has been trained based on a plurality of image pairs of one or more reference persons, wherein the plurality of image pairs includes millimeter wave and / or terahertz images of the one or more reference persons and associated visible light or infrared images of one or more persons acquired simultaneously with the millimeter wave and / or terahertz images of the one or more reference persons.

9. The device according to claim 8, wherein, The plurality of image pairs of the one or more reference persons includes image data of the one or more reference persons having different degrees of body occlusion with respect to the visible light or infrared image data.

10. A patient posture detection system (100), comprising: Medical imaging unit (110); At least one millimeter wave and / or terahertz sensor (20); And Processing unit (30); wherein the at least one millimeter wave and / or terahertz sensor is positioned near the patient on the patient support of the medical imaging unit; wherein the at least one millimeter wave and / or terahertz sensor is configured to: acquire millimeter wave and / or terahertz image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit; Wherein, the at least one millimeter-wave and / or terahertz sensor is configured to provide the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit; and Wherein, the processing unit is configured to determine the posture of the patient on the patient support of the medical imaging unit. Determining the posture of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit. The system includes a visible light or infrared sensor. The visible light or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit. The visible light or infrared sensor is configured to: before the medical imaging unit acquires the medical image data of the patient on the patient support of the medical imaging unit, acquire the visible light or infrared image data of the patient. The visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit, and determining the posture of the patient on the patient support includes using the visible light or infrared image data of the patient on the patient support of the medical imaging unit; wherein, the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and the second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame, and wherein, the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes the first visible light or infrared image of the patient on the patient support of the medical imaging unit, and the first visible light or infrared image is acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

11. A patient posture detection method (200), comprising: Before the medical imaging unit acquires the medical image data of the patient on the patient support of the medical imaging unit, at least one millimeter-wave and / or terahertz sensor positioned near the patient acquires (210) the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit; The at least one millimeter-wave and / or terahertz sensor provides (220) the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit to the processing unit; and The pose of the patient on the patient support of the medical imaging unit is determined (230) by the processing unit, wherein determining the pose of the patient on the patient support includes using the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit, wherein a visible light or infrared sensor is configured to be positioned near the patient on the patient support of the medical imaging unit, and wherein the visible light or infrared sensor is configured to: acquire visible light or infrared image data of the patient before the medical imaging unit acquires medical image data of the patient on the patient support of the medical imaging unit, wherein the visible light or infrared sensor is configured to provide the visible light or infrared image data of the patient on the patient support of the medical imaging unit to the processing unit, and wherein determining the pose of the patient on the patient support includes using the visible light or infrared image data of the patient on the patient support of the medical imaging unit; wherein the millimeter-wave and / or terahertz image data of the patient on the patient support of the medical imaging unit includes: a first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a first time frame; and a second millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit acquired at a second time frame after the first time frame, and wherein the visible light or infrared image data of the patient on the patient support of the medical imaging unit includes a first visible light or infrared image of the patient on the patient support of the medical imaging unit, the first visible light or infrared image being acquired simultaneously with the first millimeter-wave and / or terahertz image of the patient on the patient support of the medical imaging unit.

12. A computer program unit for controlling the device according to any one of claims 1-9 or for controlling the system according to claim 10, the computer program unit for controlling the device according to any one of claims 1-9 being configured to execute the method according to claim 13 when run by a processor, and the computer program unit for controlling the system according to claim 10 being configured to execute the method according to claim 13 when run by a processor.

13. A computer-readable medium storing the computer program unit according to claim 12.

Images