System and method for identifying target regions prior to organ-at-risk segmentation

Abstract

The present application provides a method and apparatus for generating a three-dimensional (3D) bounding box of a region of interest (ROI) of a patient. The method includes: receiving a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of the patient, and receiving a 2D MIP image as a sagittal view of the patient. Using the 2D MIP image, a first 2D bounding box of the patient's ROI and a second 2D bounding box of the patient's ROI are detected. A 3D MIP image of the patient is received, and a 3D bounding box of the patient's ROI is generated using the 3D MIP image, the first 2D bounding box, and the second 2D bounding box. A 3D MIP image including a 3D bounding box is provided.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 16 / 531,719 filed in the U.S. Patent and Trademark Office on August 5, 2019, the disclosure of which is incorporated herein by reference in its entirety. Background Art

[0003] Automatic organ at risk (OAR) segmentation has become a popular research topic because manual segmentation is time-consuming and operator-dependent. Recent developments in deep learning have reduced the amount of time for segmentation tasks to a few milliseconds.

[0004] However, it is usually impractical to feed large amounts of three-dimensional (3D) volume data with original size into the segmentation network due to the memory limitations of Graphics Processing Units (GPUs). To address this issue, some techniques use pruning, sliding windows, or downsampling as a preprocessing step. Downsampling may remove high-frequency information of the input image, resulting in reduced subsequent segmentation accuracy. Manual pruning introduces human interaction and is inefficient. Sliding window methods can slightly improve segmentation accuracy overall, but are more time-consuming in the inference phase.

[0005] The present disclosure effectively detects the bounding box of the head and neck, shrinks the input to the region of interest, and saves GPU memory resources while maintaining image details. Summary of the invention

[0006] According to one aspect of the present disclosure, a method for generating a three-dimensional (3D) bounding box of a region of interest (ROI) of a patient is provided, comprising: receiving, by a device, a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of the patient; receiving, by the device, a 2D MIP image as a sagittal view of the patient; detecting, by the device, a first 2D bounding box of the patient's ROI using the 2D MIP image as an axial view of the patient; detecting, by the device, a second 2D bounding box of the patient's ROI using the 2D MIP image as a sagittal view of the patient; receiving, by the device, a 3D MIP image of the patient; generating, by the device, a 3D bounding box of the patient's ROI using the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient; and providing, by the device, a 3D MIP image including the 3D bounding box of the patient's ROI to allow organ at risk (OAR) segmentation to be performed using the 3D MIP image including the 3D bounding box of the ROI.

[0007] According to one aspect of the present disclosure, a device for generating a three-dimensional (3D) bounding box of a region of interest (ROI) of a patient is provided, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate according to the instructions of the program code, wherein the program code comprises: receiving code configured to cause the at least one processor to receive a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of the patient, to receive a 2D MIP image as a sagittal view of the patient, and to receive a 3D MIP image of the patient; detecting code configured to cause the at least one processor to detect a first 2D bounding box of the ROI of the patient using the 2D MIP image as the axial view of the patient, and to detect a second 2D bounding box of the ROI of the patient using the 2D MIP image as the sagittal view of the patient; generating code configured to cause the at least one processor to generate a 3D bounding box of the ROI of the patient using the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient; and providing code configured to cause the at least one processor to provide a 3D MIP image including the 3D bounding box of the ROI of the patient, so as to allow the 3D bounding box including the 3D bounding box of the ROI to be used. MIP images are used for organ at risk (OAR) segmentation.

[0008] According to one aspect of the present disclosure, a non-transitory computer-readable medium is provided, which stores instructions, including one or more instructions, which, when executed by one or more processors of a device, cause the one or more processors to: receive a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of a patient; receive a 2D MIP image as a sagittal view of the patient; use the 2D MIP image as the axial view of the patient to detect a first 2D bounding box of an ROI of the patient; use the 2D MIP image as the sagittal view of the patient to detect a second 2D bounding box of the ROI of the patient; receive a 3D MIP image of the patient; use the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient to generate a 3D bounding box of the ROI of the patient; and provide a 3D MIP image including the 3D bounding box of the ROI of the patient to allow organ at risk (OAR) segmentation to be performed using the 3D MIP image including the 3D bounding box of the ROI. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a flow chart of an example process for generating a three-dimensional (3D) bounding box of a region of interest (ROI) of a patient using two-dimensional (2D) image data.

[0010] Figure 2 is a diagram of an example environment in which the systems and / or methods described herein may be implemented; and

[0011] Figure 3 yes Figure 2 A diagram of example components of one or more devices. DETAILED DESCRIPTION

[0012] In radiotherapy treatment planning, deep learning (DL)-based segmentation networks have been widely used for delineating the boundaries of organs at risk (OARs) and targeted tumors. However, if the input image volume has a native size (e.g., including a file size above a certain predetermined threshold), 3D-based segmentation networks often suffer from limited GPU memory resources.

[0013] The present disclosure solves this problem of 3D OAR segmentation by automatically detecting bounding boxes of the body parts of interest before the DL-based segmentation step. OAR segmentation is performed within the detected bounding boxes to reduce resource consumption. In order to reduce the memory requirements of the bounding box detection network, embodiments of the present disclosure use 2D maximum intensity projection images derived from sagittal and axial views to detect two-dimensional (2D) bounding boxes instead of using original 3D images for detection. A 3D bounding box is then constructed from the 2D bounding box in the 2D maximum intensity projection (MIP) image.

[0014] Embodiments of the present disclosure automatically detect a region of interest (ROI) from a human body (eg, the head and neck region of a computed tomography (CT) image scanned of a patient with head and neck cancer).

[0015] Embodiments of the present disclosure use 2D MIP projection images (rather than 3D images) to perform detection, thereby saving GPU memory and processing resources compared to using 3D images for bounding box detection. After identifying the bounding box of the head and neck region, embodiments of the present disclosure reduce the input volume to the head and neck region of interest. Therefore, embodiments of the present disclosure also solve the problem of GPU memory limitations in the subsequent segmentation stage.

[0016] As a specific example, the present disclosure provides an efficient way to detect ROIs (e.g., head and neck) in CT images from patients with head and neck cancer. For example, an embodiment uses 2D projection images to detect bounding boxes of the head and neck, thereby saving memory in the bounding box detection stage compared to performing detection directly using 3D image data. This is more beneficial for deep learning based OAR segmentation if the input volume for segmentation is the detected head and neck, rather than the entire volume including, for example, data associated with the bed.

[0017] Compared with detecting 2D bounding boxes, detecting 3D bounding boxes requires more GPU memory consumption. Therefore, embodiments of the present disclosure use 2D MIP images to detect bounding boxes.

[0018] Embodiments of the present disclosure may utilize fast region-based convolutional neural networks (RCNN), faster RCNN (FasterRCNN), mask RCNN (Mask RCNN), etc. for bounding box detection based on 2D deep learning.

[0019] For example, in the Faster RCNN architecture, the input image is sent to a convolutional network to extract feature maps. The feature maps are then sent to a region proposal network to predict candidate regions. The size of the proposed region is further reshaped and sent to a classifier, which classifies the information in the proposed region and predicts the actual shape of the proposed region.

[0020] In the present disclosure, embodiments include two categories (ie, axial MIP and sagittal MIP) in each proposed region. Regarding the size of the region, embodiments may predict four coordinates of each proposed region, namely, minimum x, minimum y, maximum x, and maximum y.

[0021] According to an embodiment, the 3D bounding box may include six coordinates, such as minimum x, maximum x, minimum y, maximum y, minimum z, and maximum z. The first four coordinates (i.e., minimum x, maximum x, minimum y, and maximum y) are determined based on the coordinates in the axial MIP image. In addition, the remaining coordinates (i.e., minimum z and maximum z) are determined based on the minimum x and maximum x in the sagittal image.

[0022] The present disclosure provides the following technical benefits, among others: 1) reduced GPU memory usage to allow possible and feasible ROI identification; 2) improved workflow and increased efficiency.

[0023] Figure 1 1 is a flow chart of an example process 100 for generating a three-dimensional (3D) bounding box of a region of interest (ROI) of a patient. In some implementations, the process may be performed by the platform 220. Figure 1 In some implementations, the process may be performed by another device or group of devices (eg, user device 210) that is separate from or includes platform 220. Figure 1 One or more process blocks.

[0024] like Figure 1 As shown, process 100 may include receiving, by a device, a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of a patient (block 105 ).

[0025] For example, the platform 220 may receive a 2D MIP image that is an axial view of the patient. The platform 220 may receive the 2D MIP image from another device, from a cloud storage device, etc. based on input from an operator.

[0026] A patient may refer to a person, animal, phantom, object, etc., in which a ROI is to be detected. A ROI may refer to a region in which an organ at risk (OAR) treatment is to be performed. As an example, for a patient with head and neck cancer, the ROI may include the patient's head and neck. It should be understood that the ROI may vary based on the patient. An axial view of a patient may refer to a plan view of the patient.

[0027] Further Figure 1 As shown, process 100 may include receiving, by a device, a 2D MIP image as a sagittal view of a patient (block 110 ).

[0028] Further Figure 1 As shown, process 100 may include detecting, by the device, a first 2D bounding box of a ROI of the patient using the 2D MIP image as an axial view of the patient (block 115 ).

[0029] Further Figure 1 As shown, process 100 may include detecting, by the device, a second 2D bounding box of the ROI of the patient using the 2D MIP image as a sagittal view of the patient (block 120 ).

[0030] Further Figure 1 As shown, process 100 may include receiving, by a device, a 3D MIP image of a patient (block 125 ).

[0031] Further Figure 1 As shown, process 100 may include identifying whether all data points of the first 2D bounding box and the second 2D bounding box have been obtained (block 130 ).

[0032] Further Figure 1 As shown, if all data points have not been obtained (block 130 —NO), process 100 may include obtaining additional data points.

[0033] Further Figure 1 As shown, if all data points have been obtained (block 130 —yes), process 100 may include: generating, by the device, a 3D bounding box of the patient's ROI using the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient (block 135 ).

[0034] Further Figure 1 As shown, process 100 may include providing, by a device, a 3D MIP image including a 3D bounding box of a ROI of a patient to allow organ at risk (OAR) segmentation using the 3D MIP image including the 3D bounding box of the ROI ( 140 ).

[0035] Although Figure 1Example blocks of process 100 are shown, but in some implementations, process 100 may include additional blocks, fewer blocks, different blocks, or different blocks. Figure 1 The blocks shown are blocks that are arranged differently. Additionally or alternatively, two or more blocks of process 100 may be performed in parallel.

[0036] Figure 2 2 is a diagram of an example environment 200 in which the systems and / or methods described herein may be implemented. Figure 2 As shown, environment 200 may include user device 210, platform 220, and network 230. The devices of environment 200 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0037] The user device 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with the platform 220. For example, the user device 210 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smart phone, a wireless phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), or the like. In some implementations, the user device 210 may receive information from the platform 220 and / or send information to the platform 220.

[0038] The platform 220 includes one or more devices capable of generating a three-dimensional (3D) bounding box of a patient's region of interest (ROI), as described elsewhere herein. In some implementations, the platform 220 may include a cloud server or a group of cloud servers. In some implementations, the platform 220 may be designed as a modular platform so that certain software components may be swapped in or out according to specific needs. Thus, the platform 220 may be easily and / or quickly reconfigured for different uses.

[0039] In some implementations, as shown, the platform 220 can be hosted in a cloud computing environment 222. It should be noted that although the implementations described herein describe the platform 220 as being hosted in a cloud computing environment 222, in some implementations, the platform 220 is not cloud-based (i.e., can be implemented outside of a cloud computing environment) or can be partially cloud-based.

[0040] Cloud computing environment 222 includes an environment hosting platform 220. Cloud computing environment 222 can provide computing, software, data access, storage, and other services that do not require end users (e.g., user devices 210) to know the physical location and configuration of the systems and / or devices hosting platform 220. As shown, cloud computing environment 222 may include a set of computing resources 224 (this set of computing resources is collectively referred to as "computing resources 224", and a single computing resource is referred to as "computing resource 224").

[0041] The computing resources 224 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some implementations, the computing resources 224 may control the platform 220. Cloud resources may include computing instances running in the computing resources 224, storage devices provided in the computing resources 224, data transmission devices provided by the computing resources 224, etc. In some implementations, the computing resources 224 may communicate with other computing resources 224 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0042] Further Figure 2 As shown, the computing resources 224 include a set of cloud resources, such as one or more applications ("APP") 224-1, one or more virtual machines ("VM") 224-2, virtualized storage ("VS") 224-3, one or more hypervisors ("HYP") 224-4, etc.

[0043] The applications 224-1 include one or more software applications that can be provided to or accessed by the user device 210 and / or the sensor device 220. The applications 224-1 can eliminate the need to install and run software applications on the user device 210. For example, the applications 224-1 can include software associated with the platform 220 and / or any other software that can be provided through the cloud computing environment 222. In some implementations, one application 224-1 can send information to / receive information from one or more other applications 224-1 through the virtual machine 224-2.

[0044] Virtual machine 224-2 includes a software implementation of a machine (e.g., a computer) that runs programs like a physical machine. Depending on the degree and purpose of virtual machine 224-2 to any actual machine, virtual machine 224-2 can be a system virtual machine or a process virtual machine. System virtual machines can provide a complete system platform that supports the operation of a complete operating system ("OS"). Process virtual machines can run a single program and can support a single process. In some implementations, virtual machine 224-2 can run on behalf of a user (e.g., user device 210) and can manage the infrastructure of cloud computing environment 222, such as data management, synchronization, or long-term data transfer.

[0045] Virtualized memory 224-3 includes one or more storage systems and / or one or more devices using virtualization technology within the storage system or device of computing resource 224. In some implementations, in the environment of the storage system, the type of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage, so that the storage system can be accessed without considering physical storage or heterogeneous structures. Separation allows administrators of storage systems to have flexibility in how administrators manage storage for end users. File virtualization can eliminate the dependency between data accessed at the file level and the location where the file is physically stored. This can optimize storage usage, server consolidation and / or non-interference file migration performance.

[0046] Hypervisor 224-4 may provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer such as computing resource 224. Hypervisor 224-4 may present a virtual operating platform to the guest operating systems and may manage the operation of the guest operating systems. Multiple instances of each operating system may share virtualized hardware resources.

[0047] The network 230 includes one or more wired networks and / or wireless networks. For example, the network 230 may include a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-based network, etc., and / or a combination of these or other types of networks.

[0048] Figure 2 The number and arrangement of devices and networks shown are provided as examples. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different Figure 2 The devices and / or networks shown are arranged differently. Figure 2 Two or more of the devices shown may be implemented in a single device, or Figure 2 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of environment 200 may perform one or more functions described as being performed by another set of devices of environment 200.

[0049] Figure 3 300 is a diagram of example components of device 300. Device 300 may correspond to user device 210 and / or platform 220. Figure 3As shown, device 300 may include a bus 310 , a processor 320 , a memory 330 , a storage component 340 , an input component 350 , an output component 360 , and a communication interface 370 .

[0050] The bus 310 includes components that allow communication between components of the device 300. The processor 320 is implemented in hardware, firmware, or a combination of hardware and software. The processor 320 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor 320 includes one or more processors that can be programmed to perform functions. The memory 330 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 320.

[0051] Storage component 340 stores information and / or software related to the operation and use of device 300. For example, storage component 340 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of non-transitory computer-readable medium, and a corresponding drive.

[0052] Input components 350 include components that allow device 300 to receive information, such as input through user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). In addition or alternatively, input components 350 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 360 include components that provide output information from device 300 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0053] The communication interface 370 includes transceiver-like components (e.g., a transceiver and / or a separate receiver and transmitter) that enable the device 300 to communicate with other devices, such as through a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. The communication interface 370 may allow the device 300 to receive information from another device and / or provide information to another device. For example, the communication interface 370 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0054] Device 300 may perform one or more processes described herein. Device 300 may perform these processes in response to processor 320 executing software instructions stored by a non-transitory computer-readable medium such as memory 330 and / or storage component 340. In this document, a computer-readable medium is defined as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0055] The software instructions may be read into the memory 330 and / or storage component 340 from another computer-readable medium or from another device through the communication interface 370. When executed, the software instructions stored in the memory 330 and / or storage component 340 may cause the processor 320 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0056] Figure 3 The number and arrangement of components shown are provided as examples. In practice, device 300 may include additional components, fewer components, different components, or different components. Figure 3 Additionally or alternatively, one set of components (eg, one or more components) of device 300 may perform one or more functions described as being performed by another set of components of device 300.

[0057] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Modifications and variations may be made in light of the above disclosure or may be acquired from practice of the embodiments.

[0058] As used herein, the term "component" is intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0059] Obviously, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0060] Even though particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may only be directly dependent on one claim, the disclosure of possible implementations includes each dependent claim in combination with any other claim in the claim set.

[0061] Unless explicitly described, the elements, actions or instructions used in this article should not be interpreted as critical or essential. In addition, as used herein, the articles "one" and "an" are intended to include one or more items and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related items and unrelated items, etc.), and can be used interchangeably with "one or more". In the case of only intending to indicate one item, the term "one" or similar language is used. In addition, as used herein, the terms "have", "have", "contain", etc. are intended to indicate open terms. In addition, unless otherwise expressly stated, the phrase "based on" is intended to mean "based at least in part on".

Claims

1. A method for generating a three-dimensional 3D bounding box of a region of interest (ROI) of a patient, comprising: receiving, by a device, a two-dimensional (2D) maximum intensity projection (MIP) image as an axial view of the patient; receiving, by the device, a 2D MIP image as a sagittal view of the patient; detecting, by the device, a first 2D bounding box of the ROI of the patient using the 2D MIP image as the axial view of the patient; detecting, by the device, a second 2D bounding box of the ROI of the patient using the 2D MIP image as the sagittal view of the patient; Receiving, by the device, a 3D MIP image of the patient; generating, by the device, the 3D bounding box of the ROI of the patient using the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient; removing, by the device, data of the 3D MIP image that is not within the 3D bounding box, and A 3D MIP image including the 3D bounding box of the ROI of the patient and from which the data has been removed is provided by the device to allow organ at risk (OAR) segmentation using the 3D MIP image including the 3D bounding box of the ROI.

2. The method according to claim 1, characterized in that: The method further comprises: Inputting, by the device, the 2D MIP image as the axial view of the patient and the 2D MIP image as the sagittal view of the patient into a region-based convolutional neural network (RCNN); and The device detects the first 2D bounding box and the second 2D bounding box based on the output of the RCNN.

3. The method according to any one of claims 1 to 2, characterized in that The method further comprises: A first minimum horizontal value, a first maximum horizontal value, a first minimum vertical value, and a first maximum vertical value of the first 2D bounding box of the ROI are identified by the device.

4. The method according to claim 3, characterized in that The method further comprises: A second minimum horizontal value and a second maximum horizontal value of the second 2D bounding box of the ROI are identified by the device.

5. The method according to claim 4, characterized in that The method further comprises: The 3D bounding box of the ROI is generated by the device using the first minimum horizontal value, the first maximum horizontal value, the first minimum vertical value, and the first maximum vertical value of the first 2D bounding box of the ROI and using the second minimum horizontal value and the second maximum horizontal value of the second 2D bounding box of the ROI.

6. The method according to any one of claims 1 to 2, characterized in that The ROI is the patient's head.

7. An apparatus for generating a three-dimensional 3D bounding box of a region of interest (ROI) of a patient, comprising: at least one memory configured to store program code; At least one processor is configured to read the program code and operate according to the instructions of the program code, wherein the program code includes: receiving code configured to cause the at least one processor to receive a two-dimensional 2D maximum intensity projection (MIP) image as an axial view of the patient, receive a 2D MIP image as a sagittal view of the patient, and receive a 3D MIP image of the patient; detection code configured to cause the at least one processor to detect a first 2D bounding box of the ROI of the patient using the 2D MIP image as the axial view of the patient, and to detect a second 2D bounding box of the ROI of the patient using the 2D MIP image as the sagittal view of the patient; generating code configured to cause the at least one processor to generate the 3D bounding box of the ROI of the patient using the first 2D bounding box, the second 2D bounding box, and the 3D MIP image of the patient; removing, by the device, data of the 3D MIP image that is not within the 3D bounding box, and Code is provided configured to cause the at least one processor to provide a 3D MIP image including the 3D bounding box of the ROI of the patient and from which the data has been removed to allow organ at risk (OAR) segmentation using the 3D MIP image including the 3D bounding box of the ROI.

8. The device according to claim 7, characterized in that The program code further includes: input code configured to cause the at least one processor to input the 2D MIP image as the axial view of the patient and the 2D MIP image as the sagittal view of the patient into a region-based convolutional neural network (RCNN), The detection code is further configured to enable the at least one processor to detect the first 2D bounding box and the second 2D bounding box based on the output of the RCNN.

9. The device according to any one of claims 7 to 8, characterized in that The program code further includes: Identification code configured to cause the at least one processor to identify a first minimum horizontal value, a first maximum horizontal value, a first minimum vertical value, and a first maximum vertical value of the first 2D bounding box of the ROI.

10. The device according to claim 9, characterized in that The identification code is further configured to cause the at least one processor to identify a second minimum horizontal value and a second maximum horizontal value of the second 2D bounding box of the ROI.

11. The device according to claim 10, characterized in that The generating code is further configured to cause the at least one processor to generate the 3D bounding box of the ROI using the first minimum horizontal value, the first maximum horizontal value, the first minimum vertical value, and the first maximum vertical value of the first 2D bounding box of the ROI, and using the second minimum horizontal value and the second maximum horizontal value of the second 2D bounding box of the ROI.

12. The device according to any one of claims 7 to 8, characterized in that The ROI is the patient's head.

13. A non-transitory computer-readable medium storing instructions, the instructions comprising one or more instructions, which, when executed by one or more processors of a device for generating a three-dimensional 3D bounding box of a region of interest (ROI) of a patient, cause the one or more processors to perform the method according to any one of claims 1 to 6.

14. An apparatus for generating a three-dimensional 3D bounding box of a region of interest (ROI) of a patient, comprising: a receiving module configured to receive a two-dimensional 2D maximum intensity projection MIP image as an axial view of the patient, receive a 2D MIP image as a sagittal view of the patient, and receive a 3D MIP image of the patient; a detection module configured to detect a first 2D bounding box of the ROI of the patient using the 2D MIP image as the axial view of the patient, and to detect a second 2D bounding box of the ROI of the patient using the 2D MIP image as the sagittal view of the patient; a generating module configured to generate the 3D bounding box of the ROI of the patient using the first 2D bounding box, the second 2D bounding box and the 3D MIP image of the patient; removing, by the device, data of the 3D MIP image that is not within the 3D bounding box, and A module is provided, configured to provide a 3D MIP image including the 3D bounding box of the ROI of the patient and from which the data has been removed, to allow organ at risk (OAR) segmentation using the 3D MIP image including the 3D bounding box of the ROI.

Images