Three-dimensional imaging of object on object support

Abstract

Disclosed herein is a medical system (200, 300, 500, 600) comprising: a subject support (206); a first camera (214) configured for imaging a first portion of the support surface of the object support; a second camera (216) configured for imaging a second portion of the support surface, where the first portion and the second portion comprise an overlap region, where the object support is movable relative to the first camera and the second camera along a predetermined path (218) between a first position (324), at least one intermediate position (326) and a second position (328), where the first position (324) and the at least one intermediate position (326) are different from each other, and where the second position (328) is different from the first position (324). Each of the one or more intermediate positions is located between the first position and the second position. A synthetic three-dimensional image is constructed from camera image data acquired by the first camera and the second camera for at least two positions of the object support along the predetermined path.

Description

Technical Field

[0001] The present invention relates to medical imaging, and more particularly to the configuration of a medical imaging system. Background Art

[0002] Various tomographic medical imaging techniques, such as magnetic resonance imaging (MRI), computed tomography, positron emission tomography, and single photon emission tomography, are capable of visualizing the anatomy of a subject in detail. Often, it may be beneficial to determine the position of an object before inserting it into a medical imaging system.

[0003] U.S. patent application publication US2022287669A1 discloses an automatic light arrangement for medical visualization, comprising: providing a medical 3D image; providing spatial information about a region of interest in the 3D image and spatial information about a virtual camera; determining multiple possible arrangements of light sources by using depth information based on the 3D image and spatial information about the region of interest in the image and spatial information about the virtual camera, wherein valid arrangements are those arrangements in which shadows on the region of interest are below a predefined threshold, and / or wherein the determination or arrangement is based on multiple predefined perceptual metrics specifically applied to the region of interest; prioritizing the determined arrangements; and selecting the arrangement with the best priority.

[0004] US Patent Application Publication No. US2009 / 285357A1 discloses a 3D optical system for obtaining optical and depth images of a patient as the patient moves into a bore to obtain a full body 3D mesh for identifying various body parts of the patient.

[0005] U.S. patent application publication US2019 / 394449A1 discloses a plastic surgery planning system that uses dual optics to obtain 3D images and distance information of a stationary patient.

[0006] U.S. Patent Application Publication No. US2021 / 104055A1 discloses a system that uses 3D images obtained from a capture device to check the position of a patient on a table to prepare for a medical scanning procedure. Summary of the Invention

[0007] The invention provides a medical system, a method and a computer program according to the independent claims. Embodiments are given in the dependent claims.

[0008] A difficulty in imaging an object on an object support using stereo cameras is that a pair of stereo cameras generally cannot image the entire object. Therefore, there may be blurred areas in the three-dimensional image. An embodiment may provide an improved means for providing a composite three-dimensional image. A first camera and a second camera are used to image an object on an object support that can be moved along a predetermined path between a first position, at least one intermediate position, and a second position. At the first position, the first camera captures initial first camera image data, and the second camera captures initial second camera image data. At one of the intermediate positions, the first camera captures additional first camera image data, and the second camera captures additional second camera image data. An initial three-dimensional image is reconstructed based on the initial first camera data and the initial second camera data.

[0009] Because the displacement along the predetermined path between the first position and the intermediate position is known, this information is used to reconstruct a first camera 3D image based on the initial first camera image data and the additional first camera image data. This information is also used to reconstruct a second camera 3D image based on the initial second camera image data and the additional second camera image data. A composite 3D image is then constructed by combining the initial 3D image, the first camera 3D image, and the second camera 3D image.

[0010] In one aspect, the present invention provides a medical system comprising a subject support. The subject support comprises a support surface for receiving a subject. The medical system further comprises a first camera configured to image a first portion of the support surface. The medical system further comprises a second camera configured to image a second portion of the support surface. The first portion and the second portion include an overlapping region. The subject support is movable relative to the first camera and the second camera along a predetermined path between a first position, at least one intermediate position, and a second position. Each of the one or more intermediate positions is located between the first position and the second position.

[0011] The medical system also includes a memory containing machine-executable instructions. The medical system also includes a computing system configured to control the medical system. For example, the computing system can be used to control a first camera, a second camera, and an object support. Execution of the machine-executable instructions causes the computing system to control the first camera to acquire initial first camera image data describing the object on the object support when the object support is in the first position. Execution of the machine-executable instructions also causes the computing system to control the second camera to acquire initial second camera image data describing the object on the object support when the object support is in the first position. Execution of the machine-executable instructions also causes the computing system to construct an initial three-dimensional image of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data.

[0012] Execution of the machine-executable instructions further causes the computing system to identify a region of the object imaged by the initial first camera image data but obscured by the initial second camera image data as a first occlusion region, and to identify a region of the object imaged by the initial second camera image data but obscured by the initial first camera image data as a second occlusion region. Thus, in the preceding steps, images are captured using the first camera and the second camera. These images are then used to construct an initial three-dimensional image.

[0013] Execution of the machine-executable instructions further causes the computing system to control the object support to move to one of the one or more intermediate positions at least once. Execution of the machine-executable instructions further causes the computing system to control the first camera to capture additional first camera image data depicting the object on the object support when the object support is in the one of the one or more intermediate positions at least once.

[0014] Execution of the machine-executable instructions further causes the computing system to perform at least one operation that controls the second camera to acquire additional second camera image data depicting the object on the object support when the object support is in one of the one or more intermediate positions. Execution of the machine-executable instructions further causes the computing system to construct a first camera 3D image of the object using the initial first camera image data and the additional first camera image data. Images of the object have been captured at at least two positions using a first camera. Only images from the first camera are used to construct the first camera 3D image. A potential benefit may be that one may be able to select an intermediate position where, for a given (unknown) object, the reconstruction has minimal occlusion regions, either only in regions of less interest, or with minimal occlusion in regions of interest. This is possible because occlusion regions are typically different from those that would occur if both cameras were in one position.

[0015] Execution of the machine-executable instructions further causes the computing system to construct a second camera 3D image of the object using the initial second camera image data and the additional second camera image data. Again, a potential benefit is that the multiple images acquired by the second camera are less likely to become occluded areas in the second camera 3D image.

[0016] Execution of the machine-executable instructions further causes the computing system to construct a composite 3D image, comprising the first camera 3D image and the second camera 3D image. The first occluded region is at least partially replaced by the first camera 3D image. The second occluded region is at least partially replaced by the second camera 3D image. When the object is at the first position, images from the first camera and the second camera are used to construct the initial 3D image. The first camera 3D image and the second camera 3D image can also be constructed by moving to an intermediate position and capturing additional images.

[0017] In some examples, the medical system may also have calibration data stored in memory. For example, graph paper or another object with a discernible pattern can be placed on the support surface to perform calibration. The support can then be moved from a first position to one or more intermediate positions, and images captured. This can be used to create a map of how the surface moves from the initial image to the intermediate image based on the object support. If there is an object or other object on top of the support surface, those parts of the object or object are closer to the camera than the support surface. This may result in a greater distance being moved between images than if the object were directly on the support surface and had no height.

[0018] As an alternative to calibration in memory, the extrinsic parameters of the camera relative to the medical device are known, ie a 3x3 rotation matrix and a 1x3 translation vector describing the angle and position of the camera relative to the reference coordinate system of the medical device.

[0019] This may allow the coordinates of a pixel in the camera image to be transformed into the 3D coordinates of a corresponding point in the coordinate system of the medical device.

[0020] As another alternative, if the cameras are not calibrated relative to the medical device, the height profile of the object can still be derived by processing the camera images alone, but in this case the absolute position will be unknown. However, if the imaging system or its rigid components are partially within the FOV of the first or second camera, automatic registration can be used to derive the registration.

[0021] In another embodiment, the predetermined path is a linear path.

[0022] In another embodiment, the predetermined path is a linear path, wherein the object support may be raised or lowered a predetermined amount as the object support traverses the linear path.

[0023] In another embodiment, the first camera and the second camera may have portions of the object obscured in their respective images. This may be advantageous in providing a complete three-dimensional image of the object, the object support being moved to at least one of the intermediate positions.

[0024] In another embodiment, the images used to construct the composite 3D image are stitched together to provide a more complete 3D image of the object. In some cases, portions of the images may be joined together. If areas of the object from these different images overlap, one option is to average the images to reduce the amount of noise in the composite 3D image.

[0025] In another embodiment, execution of the machine-executable instructions further causes the computing system to additionally perform at least one construction of an additional three-dimensional image of the object using the additional first camera image data and the additional second camera image data. Execution of the machine-executable instructions further causes the computing system to additionally use the additional three-dimensional image when constructing the composite three-dimensional image. This can be used, for example, to average the additional three-dimensional image with the other three-dimensional images when constructing the composite three-dimensional image. This can have the effect of reducing the amount of noise in the composite three-dimensional image, and can, for example, provide a more accurate means of constructing the composite three-dimensional image of the object.

[0026] In another embodiment, the composite 3D image includes an averaged region constructed by averaging overlapping regions between two or more of the following images: the initial 3D image, the first camera 3D image, the second camera 3D image, and the additional 3D image, as well as previous iterations of the composite 3D image. For example, as the object moves to multiple intermediate positions, additional images may become available, and these images can be averaged into the existing composite 3D image. This can provide a means of producing a very high resolution or more accurate composite 3D image of the object.

[0027] In another embodiment, the memory further comprises an anatomical keypoint module configured to output a set of anatomical keypoints of an object in response to receiving any of: the initial first camera image data, the initial second camera image data, the additional first camera image data, and the additional second camera image data. Execution of the machine-executable instructions further causes the computing system to receive initial first camera keypoint data in response to inputting the initial first camera image data into the anatomical keypoint module, and to receive initial camera keypoint data in response to inputting the initial second camera image data into the anatomical keypoint module.

[0028] This embodiment may be beneficial because superior algorithms exist for identifying anatomical keypoints in an object. By identifying these keypoints in both the first and second cameras, the stereoscopic or three-dimensional position of the keypoints may be determined. This may be particularly beneficial because it may provide an improved means of aligning various three-dimensional images to produce a composite three-dimensional image, and a means of detecting object motion or other issues during the process of acquiring the various images and constructing the composite three-dimensional image.

[0029] As used herein, anatomical key points include anatomical landmarks or locations located within the body of an object. Anatomical key points may include the locations of various joints or surface landmarks (eyes, ears, nose) of an object. A camera image describes an object and provides a description (texture and / or shape) of the exterior or surface of the object. For example, a camera image can be an optical image, an infrared image, a thermal image, or even a color image. In other examples, the camera image is a three-dimensional surface image. In another example, the camera image is a composite image formed from multiple camera images. For example, two two-dimensional images can be used to provide a stereo image that provides three-dimensional or spatial information. In other examples, the camera image is a composite image formed from multiple types of images. An infrared or thermal image can be combined with a three-dimensional image.

[0030] In another embodiment, the anatomical keypoint module includes a neural network configured to output a separate anatomical keypoint coordinate probability map for each of the at least one anatomical keypoint coordinates in response to receiving an image of the object on the object support. For example, various joints and locations within the object's body may be included in the anatomical keypoint coordinates. For each of the set, a separate image or map may be output, and the probability of the joint or anatomical location being in a particular location may be output on the map. For example, the most likely location may be obtained by taking the maximum probability. The system may be particularly good at identifying the location of joints or other anatomical locations when the joints or other anatomical structures are occluded or partially occluded. For example, if the subject has clothing or a blanket on him.

[0031] Execution of the machine-executable instructions further causes the computing system to receive a separate anatomical keypoint coordinate probability map in response to inputting the image into the neural network. Execution of the machine-executable instructions further causes the computing system to calculate the set of anatomical keypoint coordinates for each joint coordinate in the set of joint coordinates based on the separate joint coordinate probability map.

[0032] Training a neural network can be accomplished by having an image of the subject and then having a training keypoint coordinate probability map with at least one anatomical keypoint coordinate labeled for each. For example, there could be a series of images of different subjects in slightly different positions, wearing different clothing, or even covered by a blanket, or with the body partially occluded. The operator can then label the individual points to indicate the location of the anatomical keypoint coordinates. This can then be used, for example, in a deep learning algorithm to train the neural network.

[0033] In another embodiment, execution of the machine-executable instructions further causes the computing system to calculate one of one or more intermediate positions using the positions of the anatomical keypoints from the first camera keypoint data and / or using the positions of the anatomical keypoints from the initial second camera keypoint data. For example, the initial first camera keypoint data can be used to reference the position of a first occluded region, and the initial second camera keypoint data can be used to locate the position of a second occluded region. These keypoints are located in their respective images, so the positions of the occluded regions in the images can be inferred. Once the positions of these occluded regions are known, intermediate positions can be determined to obtain suitable positions for acquiring additional first camera image data and additional second camera image data.

[0034] In another embodiment, execution of the machine-executable instructions further causes the computing system to receive image-guided treatment position data describing predetermined anatomical keypoints of the subject relative to a subject support. Execution of the machine-executable instructions further causes the computing system to detect subject positioning errors by comparing the predetermined anatomical keypoints with the initial first camera keypoint data and / or the initial second camera keypoint data.

[0035] Execution of the machine executable instructions also causes the computing system to provide a warning signal when an error in object positioning is detected. For example, the warning signal can cause a user interface, an audio system, or other human interface device to provide a warning signal. In other cases, the warning signal can be used to disable the radiotherapy device if the object has moved too much. In this embodiment, the position of the object on the object support is compared with data or key points collected when the object is placed on the object support. This can help prevent positioning errors of the object during the radiotherapy process. Without the use of anatomical key points, object motion may be difficult to detect with a camera system because there are other moving parts or objects in the image and shadows that may obscure the object's position.

[0036] In another embodiment, execution of the machine-executable instructions further causes the computing system to receive additional first camera keypoint data in response to inputting the additional first camera image data into the anatomical keypoint module. Execution of the machine-executable instructions further causes the computing system to receive additional second camera keypoint data in response to inputting the additional second camera image data into the anatomical keypoint module. Execution of the machine-executable instructions further causes the computing system to detect a motion condition of the object between a first position of the object support and one of the one or more intermediate positions by detecting a difference between the initial first camera keypoint data and the additional first camera keypoint data and / or a difference between the initial second camera keypoint data and the additional second camera keypoint data.

[0037] The placement of the object support is known, so a simple translation can be used to compare the coordinates between the initial first camera keypoint data and the second additional first camera keypoint data, or to compare the differences between the initial second camera keypoint data and the additional second camera keypoint data. Execution of the machine-executable instructions also causes the computing system to exclude at least a portion of the first camera 3D image and / or the second camera 3D image from the composite 3D image when motion of the object is detected. In one example, if the object has moved, specific 3D images or image data can be discarded or excluded. In other cases, such as if the object has only moved a hand or foot, not all images need to be discarded. For example, if a specific keypoint has moved, a neighborhood around the keypoint can be identified, and image data near the keypoint can be excluded from constructing the composite 3D image of the object.

[0038] In another embodiment, the object support has a maximum width perpendicular to the predetermined path. The stereo baseline is between 10% and 200% of the maximum width of the object support. This embodiment may be beneficial because it may provide an efficient means of positioning the first camera and the second camera to obtain a high-quality three-dimensional image.

[0039] In another embodiment, the stereo base is preferably between 50% and 150% of the maximum width of the object support.Choosing the stereo base to have a width between 50% and 150% may provide an effective means of eliminating occluded areas.

[0040] In another embodiment, execution of the machine-executable instructions further causes the computing system to identify at least one stationary object region by detecting stationary voxels between the initial first camera image data and the additional first camera image data, and thereby detecting stationary voxels between the initial second camera image data and the additional second camera image data. The at least one stationary object region is excluded from the composite three-dimensional image of the object. The object support moves between the first position and the intermediate position. If an object remains stationary in images captured while the object support is in different positions, this indicates that the object is effectively stationary. Such objects may be excluded because, for example, they are not part of the object or are not moved by the object support.

[0041] In another embodiment, the medical system further includes a medical imaging system for acquiring measurement data from the imaging volume. The medical imaging system may be, for example, a tomographic medical imaging system. When the subject support is in the second position, at least a portion of the support surface is within the imaging volume. Execution of the machine-executable instructions further causes the computing system to control the subject support to move to the second position. Execution of the machine-executable instructions further causes the computing system to control the medical imaging system to acquire measurement data.

[0042] In another embodiment, execution of the machine-executable instructions further causes the computing system to reconstruct a medical image based on the measurement data.

[0043] In another embodiment, execution of the machine executable instructions further causes the computing system to calculate a scan isocenter using the synthesized three-dimensional image of the object.

[0044] In another embodiment, execution of the machine-executable instructions further causes the computing system to perform object location and orientation using the synthesized three-dimensional image.

[0045] In another embodiment, execution of the machine-executable instructions further causes the computing system to perform collision risk prediction with a medical imaging system using the synthesized three-dimensional image.

[0046] In another embodiment, if the medical system is a magnetic resonance imaging system, the computing system uses the synthesized three-dimensional image to compute a SAR estimate using the object model.

[0047] In another embodiment, execution of the machine-executable instructions further causes the computing system to calculate the subject's weight using the synthesized three-dimensional image of the subject.

[0048] In another embodiment, execution of the machine-executable instructions further causes the computing system to calculate a height of the object using the synthesized three-dimensional image of the object.

[0049] In another embodiment, the medical imaging system is a magnetic resonance imaging system.

[0050] In another embodiment, the medical imaging system is a positron emission tomography system.

[0051] In another embodiment, the medical imaging system is a single photon emission tomography system.

[0052] In another embodiment, the medical imaging system is a computed tomography system.

[0053] In another embodiment, the medical imaging system is a combined positron emission tomography and magnetic resonance imaging system.

[0054] In another embodiment, the medical imaging system is a combined positron emission tomography and computed tomography system.

[0055] In another embodiment, the medical imaging system is a combined computed tomography and radiation therapy system.

[0056] In another embodiment, the medical imaging system is a combined computed tomography and positron emission tomography system.

[0057] In another embodiment, the medical imaging system is a combined magnetic resonance imaging system and radiation therapy system.

[0058] In another embodiment, the medical imaging system is part of an image-guided radiation therapy system.

[0059] In another aspect, the present invention provides a method for controlling a medical system. The medical system includes a subject support. The subject support includes a support surface for receiving a subject. The medical system also includes a first camera configured to image a first portion of the support surface. The medical system also includes a second camera configured to image a second portion of the support surface. The first portion and the second portion include an overlapping region. The subject support is movable relative to the first camera and the second camera along a predetermined path between a first position, at least one intermediate position, and a second position. Each of the one or more intermediate positions is located between the first position and the second position.

[0060] The method includes controlling the first camera to acquire initial first camera image data depicting the object on the object support when the object support is in the first position. The method also includes controlling the second camera to acquire initial second camera image data depicting the object on the object support when the object support is in the first position. The method also includes constructing an initial three-dimensional image of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data.

[0061] The method further includes identifying, as a first occlusion region, an area of ​​the object imaged by the initial first camera image data but obscured in the initial second camera image data. The method further includes identifying, as a second occlusion region, an area of ​​the object imaged by the initial second camera image data but obscured in the initial first camera image data. Execution of the machine-executable instructions further causes the computing system to perform at least one of the following operations. This includes controlling the object support to move to the one or more intermediate positions. This also includes controlling the first camera to acquire additional first camera image data depicting the object on the object support when the object support is in one of the one or more intermediate positions. This also includes controlling the second camera to acquire additional second camera image data depicting the object on the object support when the object support is in one of the one or more intermediate positions. This also includes constructing a first camera 3D image of the object using the initial first camera image data and the additional first camera image data. This also includes constructing a second camera 3D image of the object using the initial second camera image data and the additional second camera image data. Finally, this also includes constructing a composite 3D image of the object using the initial 3D image, the first camera 3D image, and the second camera 3D image, wherein the first occluded region is at least partially replaced by the first camera 3D image, and the second occluded region is at least partially replaced by the second camera 3D image.

[0062] In another aspect, the present invention provides a computer program comprising machine-executable instructions for execution by a computing system controlling a medical system. The computer program may be stored, for example, on a non-transitory storage medium. The medical system includes an object support. The object support includes a support surface for receiving an object. The medical system also includes a first camera configured to image a first portion of the support surface. The medical system also includes a second camera configured to image a second portion of the support surface. The first portion and the second portion include an overlapping area. The object support is movable relative to the first camera and the second camera along a predetermined path between a first position, at least one intermediate position, and a second position. Each of the one or more intermediate positions is located between the first position and the second position.

[0063] Execution of the machine-executable instructions causes the computing system to control the first camera to acquire initial first camera image data depicting the object on the object support when the object support is in the first position. Execution of the machine-executable instructions also causes the computing system to control the second camera to acquire initial second camera image data depicting the object on the object support when the object support is in the first position. Execution of the machine-executable instructions also causes the computing system to construct an initial three-dimensional image of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data. Execution of the machine-executable instructions also causes the computing system to identify, as a first occlusion region, a region of the object imaged by the initial first camera image data but occluded in the initial second camera image data.

[0064] Execution of the machine-executable instructions further causes the computing system to identify, as a second occlusion region, a region of the object imaged by the initial second camera image data but occluded in the initial first camera image data. Execution of the machine-executable instructions further causes the computing system to perform at least once the following operations: controlling the object support to move to one or more intermediate positions. This further includes controlling the first camera to acquire additional first camera image data depicting the object on the object support when the object support is in one of the one or more intermediate positions. This further includes controlling the second camera to acquire additional second camera image data depicting the object on the object support when the object support is in one of the one or more intermediate positions. This further includes using the initial first camera image data and the additional first camera image data to construct a first camera 3D image of the object. This further includes using the initial second camera image data and the additional second camera image data to construct a second camera 3D image of the object. This further includes using the initial 3D image, the first camera 3D image, and the second camera 3D image to construct a composite 3D image of the object. At least the first occlusion region is at least partially replaced by the first camera 3D image. The second occluded area is at least partially replaced by the second camera 3D image.

[0065] It should be understood that one or more of the above-described embodiments of the present invention may be combined as long as the combined embodiments are not mutually exclusive.

[0066] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as an apparatus, method, or computer program product. Thus, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which are generally referred to herein as "circuits," "modules," or "systems." Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having computer executable code embodied thereon.

[0067] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" includes any tangible storage medium that can store instructions that can be executed by a processor or computing system of a computing device. A computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. A computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some embodiments, a computer-readable storage medium may also be capable of storing data that can be accessed by a computing system of a computing device. Examples of computer-readable storage media include, but are not limited to, floppy disks, magnetic hard disk drives, solid-state drives, flash memory, USB thumb drives, random access memory (RAM), read-only memory (ROM), optical disks, magneto-optical disks, and register files of a computing system. Examples of optical disks include compact disks (CDs) and digital versatile disks (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R disks. The term computer-readable storage medium also refers to various types of recording media that can be accessed by a computing device via a network or communication link. For example, data can be retrieved via a modem, the Internet, or a local area network. Computer executable code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0068] A computer-readable signal medium may include a propagated data signal embodying computer-executable code, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0069] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory directly accessible to a computing system. "Computer storage" or "storage" is another example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some embodiments, computer storage may also be computer memory, and vice versa.

[0070] As used herein, a "computing system" includes an electronic component capable of executing a program or machine-executable instructions or computer-executable code. References to computing systems including examples of a "computing system" should be interpreted as potentially including multiple computing systems or processing cores. A computing system may, for example, be a multi-core processor. A computing system may also refer to a collection of computing systems within a single computer system or distributed across multiple computer systems. The term computing system should also be interpreted as potentially referring to a collection or network of computing devices, each of which includes a processor or computing system. Machine-executable code or instructions may be executed by multiple computing systems or processors, which may be within the same computing device or even distributed across multiple computing devices.

[0071] Machine executable instructions or computer executable code may include instructions or programs that cause a processor or other computing system to perform an aspect of the present invention. The computer executable code for performing the operations of various aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as "C" programming language or similar programming languages), and compiled into machine executable instructions. In some cases, the computer executable code may be in the form of a high-level language or in a precompiled form and used in conjunction with an interpreter that generates machine executable instructions on the fly. In other cases, the machine executable instructions or computer executable code may be a programming form of a programmable logic gate array.

[0072] The computer-executable code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0073] Aspects of the present invention are described with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to an embodiment of the present invention.It should be understood that each frame or a part of a frame of a flow chart, diagram and / or block diagram can be realized by the computer program instruction of computer executable code form when applicable.It should also be understood that when not mutually exclusive, the combination of the frames in different flow charts, diagrams and / or block diagrams can be combined.These computer program instructions can be provided to the computing system of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that the instruction executed via the computing system of a computer or other programmable data processing device creates a device for realizing the function / action specified in one or more frames of a flow chart and / or block diagram.

[0074] These machine-executable instructions or computer program instructions may also be stored in a computer-readable medium, which can direct a computer, other programmable data processing apparatus or other device to operate in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0075] Machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0076] As used herein, a "user interface" is an interface that allows a user or operator to interact with a computer or computer system. A user interface may also be referred to as a human-computer interface device. A user interface can provide information or data to an operator and / or receive information or data from an operator. A user interface can enable input from an operator to be received by a computer, and can provide output from the computer to a user. In other words, a user interface can allow an operator to control or manipulate a computer, and the interface can allow the computer to indicate the effects of the operator's controls or manipulations. The display of data or information on a display or graphical user interface is an example of providing information to an operator. Receiving data via a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, game controller, webcam, headset, pedal, wired gloves, remote control, and accelerometer are all examples of user interface components that enable receiving information or data from an operator.

[0077] As used herein, a "hardware interface" includes an interface that enables a computing system of a computer system to interact with and / or control an external computing device and / or apparatus. A hardware interface can allow a computing system to send control signals or instructions to an external computing device and / or apparatus. A hardware interface can also enable a computing system to exchange data with an external computing device and / or apparatus. Examples of hardware interfaces include, but are not limited to, a universal serial bus, an IEEE 1394 port, a parallel port, an IEEE 1284 port, a serial port, an RS-232 port, an IEEE-488 port, a Bluetooth connection, a wireless LAN connection, a TCP / IP connection, an Ethernet connection, a control voltage interface, a MIDI interface, an analog input interface, and a digital input interface.

[0078] As used herein, a "display" or "display device" includes an output device or user interface suitable for displaying images or data. A display can output visual, audio, and / or tactile data. Examples of displays include, but are not limited to, computer monitors, television screens, touch screens, tactile electronic displays, and Braille screens.

[0079] Cathode ray tubes (CRTs), memory tubes, bi-stable displays, electronic paper, vectorscope displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode displays (OLEDs), projectors, and head-mounted displays.

[0080] Measurement data is defined herein as recorded measurements made by a tomographic medical imaging system describing an object. Medical imaging data can be reconstructed into a medical image. A medical image is defined herein as a reconstructed two-dimensional or three-dimensional visualization of the anatomical data contained within the medical imaging data. This visualization can be performed using a computer.

[0081] K-space data is defined herein as measurements of radio frequency signals emitted by atomic spins recorded using the antenna of a magnetic resonance device during a magnetic resonance imaging scan.Magnetic resonance data is an example of measurement data.

[0082] A magnetic resonance imaging (MRI) image or MR image is defined herein as a reconstructed two-, three- or four-dimensional visualization of anatomical data contained within the magnetic resonance imaging data. The visualization may be performed using a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0084] Figure 1 A two-dimensional rendering of a three-dimensional image is shown;

[0085] Figure 2 An example of a medical system is shown;

[0086] Figure 3 Another example of a medical system is shown;

[0087] Figure 4 Shows instructions for use Figure 3 A flowchart of a method of a medical system;

[0088] Figure 5 Another example of a medical system is shown;

[0089] Figure 6 Another example of a medical system is shown; and

[0090] Figure 7 Shows instructions for use Figure 6 Flowchart of a method of a medical system.

[0091] Reference Signs List

[0092] 100 3D images

[0093] 102 Occlusion Regions of 3D Images

[0094] 200 Medical System

[0095] 202 Top view

[0096] 204 Side View

[0097] 206 Object Support

[0098] 208 support surface

[0099] 210 objects

[0100] 212 Medical Imaging Systems

[0101] 212'Magnetic Resonance Imaging System

[0102] 214 First Camera

[0103] 216 Second Camera

[0104] 218 Predetermined Path

[0105] 220 baseline

[0106] 222 First occlusion area

[0107] 224 Second occlusion area

[0108] 226 Width of the object support

[0109] 300 Medical System

[0110] 304 actuator

[0111] 312 Computer

[0112] 314 Hardware Interface

[0113] 316 Computing Systems

[0114] 318 User Interface

[0115] 320 memory

[0116] 322 Height above support surface

[0117] 324 Position of the object support at the first position

[0118] 326 Position of the object support at the second position

[0119] 328 Object support position in the middle position

[0120] 330 Displacement of the object support between the first position and the second position

[0121] 332 Displacement of the object support between the first position and the intermediate position

[0122] 340 machine-executable instructions

[0123] 342 Initial first camera image data

[0124] 344 Initial second camera image data

[0125] 346 Initial 3D Image

[0126] 348 Additional first camera image data

[0127] 350 Additional second camera image data

[0128] 352 First Camera 3D Image

[0129] 354 Second camera 3D image

[0130] 356 Additional 3D Image

[0131] 358 Synthetic 3D Image

[0132] 360 control signal

[0133] 400 Controlling a first camera to acquire initial first camera image data describing an object on an object support when the object support is in a first position

[0134] 402 Controlling a second camera to acquire initial second camera image data depicting an object on the object support when the object support is in the first position

[0135] 404 constructs an initial three-dimensional image of the object using at least a portion of an overlapping region of the initial first camera image data and the initial second camera image data.

[0136] 406 Identify a region of the object imaged by the initial first camera image data but occluded in the initial second camera image data as a first occlusion region

[0137] 408 Identify a region of the object imaged by the initial second camera image data but occluded in the initial first camera image data as a second occlusion region

[0138] 410 Controlling the object support to move to one of the one or more intermediate positions

[0139] 412 controlling the first camera to acquire additional first camera image data describing the object on the object support when the object support is in one of the one or more intermediate positions

[0140] 414 controlling the second camera to acquire additional second camera image data describing the object on the object support when the object support is in one of the one or more intermediate positions

[0141] 416 constructs a first camera three-dimensional image of the object using the initial first camera image data and the additional first camera image data

[0142] 418 constructs a second camera three-dimensional image of the object using the initial second camera image data and the additional second camera image data

[0143] 420 constructing an additional three-dimensional image of the object using the additional first camera image data and the additional second camera image data

[0144] 422 Constructing a synthetic three-dimensional image of an object

[0145] 500 Medical Systems

[0146] 504 Imaging Volume

[0147] 506 Medical Imaging System Bore

[0148] 540 measurement data

[0149] 540'k spatial data

[0150] 542 Medical Images

[0151] 542'Magnetic resonance imaging

[0152] 600 Medical Systems

[0153] 604 main magnet

[0154] 608 Field of View

[0155] 610 Magnetic Field Gradient Coil

[0156] 612 Gradient Coil Power Supply

[0157] 614 Coil

[0158] 615 Body Coil

[0159] 616 transceiver

[0160] 640 Pulse sequence command

[0161] 700 Control the object support to move to the second position

[0162] 702 Controlling a medical imaging system to acquire measurement data

[0163] 704 preferably reconstructs a medical image based on the measurement data DETAILED DESCRIPTION

[0164] Elements with the same number in these figures are equivalent elements or perform the same function. Elements discussed previously may not necessarily be discussed in later figures if the function is equivalent.

[0165] Stereo-based depth sensing, including structured light enhancement, is affected by shadows in areas where neither camera has a line of sight. Typically, these problem areas occur at the edges of objects. This effect is as follows Figure 1 shown.

[0166] Figure 1 An example of a two-dimensional rendering of a three-dimensional image is shown. The image shows a three-dimensional image of a cat. Two black bands 102 are visible on either side of the cat. These black bands are occlusion regions 102. These occlusion regions 102 are invisible to both cameras. Therefore, it is impossible to construct a three-dimensional image 100 within these occlusion regions 102. These are essentially holes in the three-dimensional image.

[0167] To overcome this problem, AI-based algorithms have recently been proposed to fill the resulting data gaps with plausible values. The drawback of this approach is that these values ​​aren't actually measured and therefore could be erroneous. A similar problem arises in external areas where the FOVs of the two cameras don't overlap. In these cases, there's simply no stereo data to match from one camera to the other. The effect is a reduced FOV relative to the native FOV of each camera in the stereo pair.

[0168] These well-known shortcomings of stereo vision, while generally difficult to address for robotic navigation purposes, can be overcome in the context of medical imaging examinations involving a patient support that can be automatically moved in a controlled manner along a defined axis. Consequently, no additional sensors are required to fill in data gaps with real-world data. Instead, the table motion axis is used to acquire additional images from each of the two cameras. Using these additional images, along with the known table displacement between image acquisitions, allows data gaps to be filled and a dense depth map with the same resolution as the cameras' intrinsic resolution to be generated. A key insight and trick here is to select any two suitable time points [tstart, tend] from any table travel event. Such a table travel event is a phase where the patient is immobilized or stable, and the examination setup does not change. Therefore, depth errors due to motion are minimized. Several such events may occur during exam preparation, at least one of which is the insertion of the table from its upper or lower position.

[0169] Figure 2 A simplified diagram of a medical system 200 is shown. Figure 2 Not all components of medical system 200 are shown. Top view 202 and side view 204 are shown adjacent to each other. Medical system 200 is shown as including a subject support 206 having a support surface 208. Subject 210 is positioned on support surface 208. Subject support 206 is shown in a first position and is movable along a predetermined path 218 to position subject 210 within medical imaging system 212. A first camera 214 and a second camera 216 are positioned above support surface 208 to image the top of subject 210 and the exposed portion of support surface 208. A baseline 220 perpendicular to the predetermined path exists between cameras 214 and 216. On subject 210, a first obstruction region 222 is an area that can be imaged by the first camera but not the second camera 216. A second obstruction region 224 exists that can be imaged by the second camera 216 but not the first camera 214. Subject support 206 has a width 226. The width of baseline 220 may be defined relative to the width of object support 226. In side view 204, the views of first camera 214 and second camera 216 overlap.

[0170] As can be seen, when the object support 206 is in the first position, the occluded areas 222 and 224 cannot be made into the three-dimensional image. However, if the object support 206 is moved along the predetermined path 218 and multiple images are acquired, these multiple images from the first camera 214 or the second camera 216 can be used to construct additional three-dimensional images that can be used to fill in the occluded areas 222, 224.

[0171] The two cameras 214, 216 are connected to a computing system that can be connected to the MR scanning (or other imaging modality) hardware and software. The computing system receives the table position and the camera images. The cameras 214, 216 also receive the table motion state or position and can select any number of camera images for a given table travel event within the time interval [tstart, tend]. The disparity between pairs of images during the event is calculated. The image frames can be adjacent or at any point in time during the table travel. A neural network can optionally be used to determine suitable frame candidates, where the optical flow in the image is limited to the optical flow of coherent table motion. For example, software for determining the location of anatomical key points can be used for this determination.

[0172] The travel distance between any two acquired images can be used to convert the calculated disparity into a depth map. Stationary portions of the scene have the most robust stereo matching performance and result in infinite distance. These regions are filtered out. Additionally, the table area can be calibrated, and the disparity calculation can be restricted to the table area. Multiple image frames during table travel can be used to identify incoherent motion or stationary obstacles. Disparity information from multiple image pairs can be used to compose a dense disparity map by removing spurious incoherent motion regions or stationary obstacles and filling in information with reliable values ​​from other image pairs when needed. When coupled to a patient keypoint detection AI network, this gap filling can be focused on the examination target area or specific parts of the examination setup. Furthermore, the patient keypoint detection network can be used to detect patient motion events during table travel, for example, by comparing relative keypoint positions between selected camera images. This can help ensure that patient motion does not corrupt the results of the proposed algorithm.

[0173] A fundamental advantage of this approach over stereo matching with a static baseline is that there are multiple baseline measurements for each image patch. This allows cross-validation of stereo matching confidence values, noise reduction by averaging, or selection of the most reliable measurements from a series of images. Any number of matching computations can be elegantly parallelized using current GPU architectures.

[0174] Figure 3An example of a medical device 300 is shown. The medical device is shown as including a subject support 206. The subject support 206 includes an actuator 304 configured to move the subject support 206 a controlled distance or displacement along a predetermined path 218. A subject 210 lies on a support surface 208. The support surface 208 faces two cameras 214, 216. A side view 204 is shown so that the views of the two cameras 214, 216 overlap. The cameras 214, 216 can capture images of the support surface 208 and / or the subject 210 when the subject support 206 is in various positions. The medical device 300 is also shown as including a computer 312. The computer 312 includes a hardware interface 314 that enables a computing system 316 to communicate with and control the other components of the medical device 300.

[0175] Specifically, in this figure, the hardware interface 314 is shown as being connected to two cameras 314, 316 for acquiring images, and to the actuator 304 for controlling the position of the object support 206. In other examples or embodiments, the hardware interface 314 can be used to control additional components, such as the medical imaging system 212 (or scanner).

[0176] The computing system 316 communicates with the hardware interface 314, memory 320, and an optional user interface 318. The memory 320 can be any combination of memories accessible to the computing system 316. This can include memory such as main memory, cache, and non-volatile memory such as flash RAM, a hard drive, or other storage devices. In some examples, the memory 320 can be considered a non-transitory computer-readable medium.

[0177] Arrow 322 indicates the height or distance above support surface 208. Dashed line 324 indicates the position of the edge of object support 206 when the object support is in first position 324. Dashed line 330 indicates the position of the edge of object support 206 when object support 206 is in second position 330. Dashed line 328 shows the current position of the edge of object support 206. Object support 206 is currently in intermediate position 328. Images are captured using cameras 214, 216 as object support 206 moves to different displacements 332, 334 relative to first position 324. Portions of object 210 that are closer to cameras 214, 216 than support surface 208 may have moved more than support surface 208 within multiple images.

[0178] For example, a graphic or other pattern can be placed on the support surface 208, and the support surface can be imaged at multiple locations. This can provide information about how the displacements 332, 334 relate to the pixel displacements of the image of the support surface 208. When the object 210 is placed on the support surface 208, the pixels representing the same portion of the object 210 will move by a greater amount than if the support surface 208 were moved alone. This greater movement of individual pixels or groups of pixels can be used to make a three-dimensional measurement of the distance 322 of the surface of the object 210 above the support surface 208.

[0179] Memory 320 is shown as containing machine-executable instructions 340. The machine-executable instructions enable computing system 316 to perform various computing tasks, such as computing a three-dimensional image from a number of two-dimensional images. Memory 320 is also shown as containing initial first camera image data 342 and initial second camera image data 344. Memory 320 is also shown as containing an initial three-dimensional image 346 computed from initial first camera image data 342 and initial second camera image data 344. After object support 102 is moved to intermediate position 128, several additional images are acquired. Memory 320 is also shown as containing additional first camera image data 348 and additional second camera image data 350. Memory 320 is also shown as containing a first camera three-dimensional image 352 computed from initial first camera image data 342 and additional first camera image data 348.

[0180] Memory 320 is also shown as containing a second camera 3D image 354 calculated from initial second camera image data 344 and additional second camera image data 350. Memory 320 is also shown as optionally containing additional 3D images calculated from first camera 3D image 352 and second camera 3D image 354. Memory 320 is also shown as containing a composite 3D image 358 calculated from initial 3D image 346, first camera 3D image 352, second camera 3D image 354, and optional additional 3D image 356. Composite 3D image 358 may be constructed by stitching together various portions of the various 3D images 346, 352, 354, and 356. In cases where some images overlap, they are averaged. Memory 320 is also shown as containing an optional control signal 360 generated using composite 3D image 358. For example, composite 3D image 358 can be used for things like locating an object, determining its mass, determining whether SAR is a problem in magnetic resonance imaging, and even detecting the probability of a collision between object 210 and the medical imaging system.

[0181] Figure 4 The diagram shows the operation Figure 3 Flowchart of the method of the medical system 300. The method is also applicable to Figure 2The medical system 200 in FIG. First, in step 400, the first camera 214 is controlled to acquire initial first camera image data 342 when the object support 206 is in the first position 324. Next, in step 402, the second camera 216 is controlled to acquire initial second camera image data 344 when the object support 206 is in the first position 324. Next, in step 406, an initial three-dimensional image 346 is constructed using the overlapping area of ​​the initial first camera image data 342 and the initial second camera image data 344.

[0182] Next, in step 406, first occlusion region 222 is identified in initial second camera image data 344. Next, in step 208, second occlusion region 224 is identified in initial first camera image data 342. The following steps 410, 412, 414, 416, 418, 420, and 422 are repeated for at least one intermediate position. In step 410, the object support is controlled to move to one of the one or more intermediate positions 328. Then, in step 412, first camera 414 is controlled to acquire additional first camera image data 348. Next, in step 414, second camera 216 is controlled to acquire additional second camera image data 350. Next, in step 416, first camera 3D image 352 is constructed based on initial first camera image data 342 and additional first camera image data 348. Next, in step 418, second camera 3D image 354 is constructed based on initial second camera image data 344 and additional second camera image data 350. Step 420 is optional. In step 420, additional 3D image 356 is constructed based on additional first camera image data 348 and additional second camera image data 350. Finally, in step 422, composite 3D image 358 of the object is constructed using initial 3D image 346, first camera 3D image 352, and second camera 3D image 354. In some examples, composite 3D image 358 may also be constructed using additional 3D image 356. First occluded region 222 is at least partially replaced by first camera 3D image 352. Second occluded region 224 is at least partially replaced by second camera 3D image 354. Steps 410, 412, 414, 416, 418, 420, and 422 may be repeated to reduce the size of occluded regions 222 and 224 and / or reduce noise in composite 3D image 358.

[0183] Figure 5 Another example of a medical device 500 is shown. The medical device in Figure 500 is similar to Figure 2The medical device 200 in FIG. 2 is shown, except that a medical imaging system 212 is now additionally included. The medical imaging system has a medical imaging volume 504 from which measurements can be acquired. The subject support 206 is configured to move at least a portion of the subject 210 into the medical imaging volume 504. In this example, the medical imaging system 212 is cylindrical and has a bore 506 into which the subject 210 can be moved using the subject support 206. However, this is not required, and not all medical imaging systems 212 need to have the cylindrical symmetry shown. Additionally, it should be noted that the camera 110 is now mounted to the medical imaging system 302 and is aligned at an oblique angle with respect to the support surface 108. The memory 320 is also shown as containing measurement data 540 acquired while the subject 210 was at least partially within the imaging volume 504. The memory 320 is further shown as containing a medical image 542 that has been reconstructed based on the measurement data 540.

[0184] Figure 6 Another example of a medical device 600 is shown. Figure 6 The medical device 600 is similar to Figure 5 . However, in this example, the medical imaging system 212 is a magnetic resonance imaging system 212'. The magnetic resonance imaging system 212' includes a main magnet 604, which can be referred to as a magnet. The magnet 404 is a superconducting cylindrical magnet 604 having a bore 506 extending therethrough. Different types of magnets can also be used. Inside the cryostat of the cylindrical magnet, there is a set of superconducting coils. Within the bore 506 of the cylindrical magnet 604 is an imaging volume 504, where the magnetic field is sufficiently strong and uniform to perform magnetic resonance imaging.

[0185] Within the bore 506 of the magnet, there is also a set of magnetic field gradient coils 610 for acquiring magnetic resonance data to spatially encode magnetic spins within the imaging volume 504 of the magnet 604. The magnetic field gradient coils 610 are connected to a magnetic field gradient coil power supply 612. The magnetic field gradient coils 610 are intended to be representative. Typically, the magnetic field gradient coils 610 include three separate sets of coils for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 610 is controlled as a function of time and can be ramped or pulsed.

[0186] Adjacent to the imaging volume 504 is a magnetic resonance coil or antenna 614, which serves as a radio frequency antenna for manipulating the orientation of magnetic spins within the imaging volume 504 and for receiving radio transmissions from spins also within the imaging volume 504. The radio frequency coil can also be comprised of multiple coil elements. An radio frequency antenna can also be referred to as a channel. The coil 614 is connected to a radio frequency transceiver 616. The coil 614 and radio frequency transceiver 616 can have separate transmitters and receivers. The coil 614 and transceiver 616 form a radio frequency system.

[0187] If coil 614 is made of multiple coil elements, they can be used to acquire magnetic resonance data individually. Thus, the coil elements can be used in parallel imaging magnetic resonance techniques. An optional body coil 615 is also shown. Body coil 615 is useful in parallel imaging techniques because it can acquire data simultaneously with the individual coil elements and use it to calculate a set of coil sensitivities. Magnetic resonance data can be acquired from within imaging volume 504. Magnetic resonance data is an example of medical image data.

[0188] Within the bore 506 of the magnet 604, the subject support 206 is shown supporting a portion of the subject 210 in the imaging volume 504. The subject support 206 is in the second position 330.

[0189] The transceiver 616, the actuator of the object support 604, the two cameras 214, 216, the actuator 304, and the gradient controller 612 are shown as connected to the hardware interface 314 of the computer system 312. The machine-executable instructions 140 are located within the memory 320.

[0190] The computer memory 320 is also shown as containing pulse sequence commands 640. The pulse sequence commands 640 are instructions or data that can be converted into instructions that can be used to control the magnetic resonance imaging system 212' to acquire measurement data 540. In this case, the measurement data 540 is k-space data 540'. In this example, the medical image 542 is a magnetic resonance image 542'.

[0191] Within the imaging volume 504 exists a field of view 608. For example, before the object is moved to the second position 330, the field of view 608 can be identified using the synthesized three-dimensional image 358.

[0192] Figure 7 Shows the description and Figure 4 The method is similar to the method shown in the flowchart of the method. Figure 7In the illustrated method, method steps 400-422 are first performed. After step 422, step 600 is performed. In step 600, the subject support 206 is moved to the second position 330. Next, in step 602, the medical imaging system 212 is controlled to acquire measurement data 642. In this example, the measurement data 642 is k-space data. Finally, in step 604, the measurement data 642 is reconstructed into a medical image 644. In this example, the medical image would be a magnetic resonance image 644.

[0193] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0194] Other variations of the disclosed embodiments may be understood and implemented by those skilled in the art in practicing the claimed invention by studying the drawings, the disclosure and the claims. In the claims, the word "comprising" does not exclude other elements or steps, and the quantifier "a" or "an" does not exclude a plurality. A single processor or other unit may perform the functions of several items recited in the claims. The fact that certain measures are cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or solid-state medium provided with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A medical system (200, 300, 500, 600), comprising: an object support (206), wherein the object support comprises a support surface (208) for receiving an object (210), a first camera (214) configured to image a first portion of the support surface; a second camera (216) configured to image a second portion of the support surface, wherein the first portion and the second portion include an overlapping region, wherein the object support is movable relative to the first camera and the second camera along a predetermined path (218) between a first position (324), at least one intermediate position (326), and a second position (328), wherein each of the one or more intermediate positions is located between the first position and the second position; a memory (320) containing machine-executable instructions (340); and A computing system (316) configured to control the medical system, wherein execution of the machine-executable instructions causes the computing system to: controlling (400) the first camera to acquire initial first camera image data (342) depicting the object on the object support when the object support is in the first position; controlling (402) the second camera to acquire initial second camera image data (344) depicting the object on the object support when the object support is in the first position; constructing (404) an initial three-dimensional image (346) of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data; identifying (406) a region of the object imaged by the initial first camera image data but occluded in the initial second camera image data as a first occlusion region (222); and identifying (408) a region of the object imaged by the initial second camera image data but occluded in the initial first camera image data as a second occlusion region (224); The execution of the machine-executable instructions further causes the computing system to perform the following operations at least once: controlling (410) the object support to move to one of the one or more intermediate positions; controlling (412) the first camera to acquire additional first camera image data (348) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; controlling (414) the second camera to acquire additional second camera image data (350) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; constructing (416) a first camera three-dimensional image (352) of the object using the initial first camera image data and the additional first camera image data; constructing (418) a second camera three-dimensional image (354) of the object using the initial second camera image data and the additional second camera image data; A composite three-dimensional image (358) of the object is constructed (422) using the initial three-dimensional image, the first camera three-dimensional image, and the second camera three-dimensional image, wherein the first occluded area is at least partially replaced by the first camera three-dimensional image, and wherein the second occluded area is at least partially replaced by the second camera three-dimensional image.

2. The medical system according to claim 1, wherein Execution of the machine-executable instructions further causes the computing system to additionally perform at least once the following operations: construct (420) an additional three-dimensional image (356) of the object using the additional first camera image data and the additional second camera image data, and wherein the composite three-dimensional image is additionally constructed using the additional three-dimensional image of the object.

3. The medical system according to claim 2, wherein: The synthesized 3D image includes an average region constructed by averaging overlapping regions between two or more of the following images: the initial 3D image, the first camera 3D image, the second camera 3D image, the additional 3D image, and a previous iteration of the synthesized 3D image.

4. The medical system according to any one of the preceding claims, wherein The memory further includes an anatomical keypoint module configured to output a set of anatomical keypoints of the object in response to receiving any of: the initial first camera image data, the initial second camera image data, the additional first camera image data, and the additional second camera image data, wherein execution of the machine-executable instructions further causes the computing system to: receiving initial first camera keypoint data in response to inputting the initial first camera image data into the anatomical keypoint module; and Initial second camera keypoint data is received in response to inputting the initial second camera image data into the anatomical keypoint module.

5. The medical system according to claim 4, wherein Execution of the machine-executable instructions further causes the computing system to calculate the one of the one or more intermediate positions using positions of anatomical keypoints of the initial first camera keypoint data and / or using positions of anatomical keypoints of the initial second camera keypoint data.

6. The medical system according to claim 4 or 5, wherein: Execution of the machine-executable instructions further causes the computing system to: receiving image-guided treatment position data describing predefined anatomical keypoints of the subject relative to the subject support; detecting object positioning errors by comparing the predefined anatomical keypoints with the initial first camera keypoint data and / or the initial second camera keypoint data; and A warning signal is provided when the object positioning error is detected.

7. The medical system according to claim 4, 5 or 6, wherein: Execution of the machine-executable instructions further causes the computing system to: receiving additional first camera keypoint data in response to inputting the additional first camera image data into the anatomical keypoint module; receiving additional second camera keypoint data in response to inputting the additional second camera image data into the anatomical keypoint module; detecting an object motion condition between the first position of the object support and the one of the one or more intermediate positions by detecting a difference between the initial first camera keypoint data and the additional first camera keypoint data and / or a difference between the initial second camera keypoint data and the additional second camera keypoint data; At least a portion of the first camera 3D image and / or the second camera 3D image is excluded from the composite 3D image upon detection of the object motion condition.

8. The medical system according to any one of the preceding claims, wherein The first camera and the second camera form a stereo baseline (220) perpendicular to the predetermined path.

9. A medical system according to any one of claims 8, wherein The object support has a maximum width (226) perpendicular to the predetermined path, wherein the stereo baseline is between 10% and 200% of the maximum width of the object support, wherein the stereo baseline is preferably between 50% and 150% of the maximum width of the object support.

10. The medical system according to any one of the preceding claims, wherein Execution of the machine-executable instructions further causes the computing system to identify at least one stationary object region by detecting stationary voxels between the initial first camera image data and the additional first camera image data and / or by detecting stationary voxels between the initial second camera image data and the additional second camera image data, and wherein the at least one stationary object region is excluded from the synthesized three-dimensional image of the object.

11. The medical system according to any one of the preceding claims, wherein The medical system also includes a medical imaging system (212, 212') for acquiring measurement data (540, 540') from an imaging volume (504), wherein when the object support is in the second position, at least a portion of the support surface is within the imaging volume, wherein execution of the machine-executable instructions further causes the computing system to: controlling (700) the object support to move to the second position; controlling (702) the medical imaging system to acquire the measurement data; and A medical image (542, 542') is preferably (704) reconstructed from the measurement data.

12. The medical system according to claim 10 or 11, wherein: Execution of the machine-executable instructions further causes the computing system to perform any one of the following: calculating the isocenter of the object, determining the position and / or orientation of the object, performing a collision risk prediction of the object with the medical imaging system, calculating a SAR estimate of the object, calculating the object's weight, calculating the object's height, and combinations thereof.

13. The medical system of claim 10, 11 or 12, wherein: The medical imaging system is any one of the following systems: a magnetic resonance imaging system (212'), a positron emission tomography system, a single photon emission tomography system, a computed tomography system, a positron emission tomography and magnetic resonance imaging combined system, a positron emission tomography and computed tomography combined system, a computed tomography and radiotherapy combined system, a computed tomography and positron emission tomography combined system, a magnetic resonance imaging system and radiotherapy combined system, and an image-guided radiotherapy system.

14. A method of controlling a medical system (200, 300, 500, 600), wherein: The medical system comprises: an object support (206), a first camera (214), and a second camera (216), wherein the object support comprises a support surface (208) for receiving an object (210); the first camera is configured to image a first portion of the support surface; the second camera is configured to image a second portion of the support surface, wherein the first portion and the second portion include an overlapping area, wherein the object support is movable relative to the first camera and the second camera along a predetermined path (218) between a first position (324), at least one intermediate position (326), and a second position (328), wherein each of the one or more intermediate positions is located between the first position and the second position; The method comprises: controlling (400) the first camera to acquire initial first camera image data (342) depicting the object on the object support when the object support is in the first position; controlling (402) the second camera to acquire initial second camera image data (344) depicting the object on the object support when the object support is in the first position; constructing (404) an initial three-dimensional image (346) of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data; identifying (406) a region of the object imaged by the initial first camera image data but occluded in the initial second camera image data as a first occlusion region (222); and identifying (408) a region of the object imaged by the initial second camera image data but occluded in the initial first camera image data as a second occlusion region (224); The execution of the machine-executable instructions further causes the computing system to perform the following operations at least once: controlling (410) the object support to move to one of the one or more intermediate positions; controlling (412) the first camera to acquire additional first camera image data (348) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; controlling (414) the second camera to acquire additional second camera image data (350) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; constructing (416) a first camera three-dimensional image (352) of the object using the initial first camera image data and the additional first camera image data; constructing (418) a second camera three-dimensional image (354) of the object using the initial second camera image data and the additional second camera image data; A composite three-dimensional image (358) of the object is constructed (422) using the initial three-dimensional image, the first camera three-dimensional image, and the second camera three-dimensional image, wherein the first occluded area is at least partially replaced by the first camera three-dimensional image, and wherein the second occluded area is at least partially replaced by the second camera three-dimensional image.

15. A computer program comprising machine executable instructions (340) for execution by a computing system (326) for controlling a medical system (200, 300, 500, 600), wherein The medical system comprises: an object support (206), a first camera (214), and a second camera (216), wherein the object support comprises a support surface (208) for receiving an object (210); the first camera is configured to image a first portion of the support surface; the second camera is configured to image a second portion of the support surface, wherein the first portion and the second portion include an overlapping area, wherein the object support is movable relative to the first camera and the second camera along a predetermined path (218) between a first position (324), at least one intermediate position (326), and a second position (328), wherein each of the one or more intermediate positions is located between the first position and the second position; wherein execution of the machine-executable instructions causes the computing system to: controlling (400) the first camera to acquire initial first camera image data (342) depicting the object on the object support when the object support is in the first position; controlling (402) the second camera to acquire initial second camera image data (344) depicting the object on the object support when the object support is in the first position; constructing (404) an initial three-dimensional image (346) of the object using at least a portion of the overlapping region of the initial first camera image data and the initial second camera image data; identifying (406) a region of the object imaged by the initial first camera image data but occluded in the initial second camera image data as a first occlusion region (222); and identifying (408) a region of the object imaged by the initial second camera image data but occluded in the initial first camera image data as a second occlusion region (224); The execution of the machine-executable instructions further causes the computing system to perform the following operations at least once: controlling (410) the object support to move to one of the one or more intermediate positions; controlling (412) the first camera to acquire additional first camera image data (348) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; controlling (414) the second camera to acquire additional second camera image data (350) depicting the object on the object support when the object support is in the one of the one or more intermediate positions; constructing (416) a first camera three-dimensional image (352) of the object using the initial first camera image data and the additional first camera image data; constructing (418) a second camera three-dimensional image (354) of the object using the initial second camera image data and the additional second camera image data; A composite three-dimensional image (358) of the object is constructed (422) using the initial three-dimensional image, the first camera three-dimensional image, and the second camera three-dimensional image, wherein the first occluded area is at least partially replaced by the first camera three-dimensional image, and wherein the second occluded area is at least partially replaced by the second camera three-dimensional image.

Images