Image processing system and method

Abstract

The present application provides a system and method for processing a 3D image. The method may include obtaining at least two slices in a direction of the 3D image from the 3D image; processing the at least two slices in the direction using a transformation model; and determining a transformed 3D image in the direction based on the at least two processed slices in the direction.

Description

[0001] Cross-references

[0002] This application claims priority to U.S. application No. US17 / 105,613 filed on November 26, 2020, the contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to a system and method for processing an image, and in particular to a system and method for processing a 3D image. Background Art

[0004] Image processing, especially 3D image processing, is widely used. In many cases, it can be more convenient, more efficient, and more effective to process 3D images on a slice basis using a processing method for 2D images. For example, prior art techniques for processing 3D images typically use a neural network trained with 3D sample images. However, compared to designing a neural network for 2D images, training a neural network for 3D images may be more difficult, and the training and use of such a neural network typically consumes a large amount of computing resources. Therefore, it is desirable to provide a system and method for processing 3D images that can reduce computing resources and improve its performance while achieving good 3D image quality. Summary of the invention

[0005] On the one hand, an embodiment of the present application discloses a system for processing a 3D image, comprising: at least one storage device comprising a set of instructions for processing a 3D image; and at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to perform operations, the operations comprising: obtaining at least two slices in a direction of the 3D image from the 3D image; processing the at least two slices in the direction using a transformation model to obtain the at least two processed slices in the direction, and determining a transformed 3D image in the direction based on the at least two processed slices.

[0006] In some embodiments, the operations further include acquiring at least one second slice of the direction to be displayed based on the at least two processed slices.

[0007] In some embodiments, the operation further includes: acquiring at least two slices in a second direction of the 3D image from the 3D image along the second direction; processing the at least two slices in the second direction using a second transformation model to acquire at least two processed slices in the second direction, and determining a transformed 3D image in the second direction based on the at least two processed slices.

[0008] In some embodiments, the operations include determining a transformed 3D image of the second direction based on the at least two processed slices of the second direction.

[0009] In some embodiments, the transformation model is different from the second transformation model.

[0010] In some embodiments, the direction is different from the second direction.

[0011] In some embodiments, the 3D image may be a CBCT image.

[0012] In some embodiments, the conversion model can be obtained by the following process: acquiring at least two training samples, each training sample comprising a sample MSCT image and a sample CBCT image, and acquiring the conversion model by training an accident model based on the at least two training samples.

[0013] In some embodiments, the conversion model is an image style transfer model.

[0014] In some embodiments, based on the at least two training samples, training the preliminary model includes: for each of the at least two training samples, acquiring at least two sample CBCT slices in the direction from the sample CBCT image of the training sample; acquiring at least two sample MSCT slices in the direction from the sample MSCT image of the training sample; and training the preliminary model using the at least two sample CBCT slices in the direction as input and the at least two sample MSCT slices in the direction as expected output.

[0015] In some embodiments, for at least one of the at least two training samples, the sample CBCT image of the training sample is determined by applying a simulation operation on the sample MSCT image of the training sample.

[0016] In some embodiments, the simulation operation includes: converting the sample MSCT image into at least one 3D array; obtaining projection values ​​of the sample MSCT image by executing a projection algorithm and a noise simulation algorithm on the sample MSCT image based at least in part on the at least one 3D array; and reconstructing the projection values ​​by using a reconstruction algorithm to obtain the sample CBCT image.

[0017] In some embodiments, the direction is associated with at least one of an axial plane, a coronal plane, or a sagittal plane of the 3D image.

[0018] In some embodiments, the conversion model is a single conversion model for processing slices in at least two directions, and the single conversion model is obtained by the following process: training a preliminary model using at least two sample slices in the at least two directions.

[0019] In some embodiments, the at least one processor is configured such that the system further performs the operation further comprising: obtaining at least two slices in a second direction of the 3D image from the 3D image along the second direction; and processing the at least two slices in the second direction using the transformation model to obtain at least two processed slices in the second direction, wherein the direction is different from the second direction.

[0020] On the other hand, some embodiments of the present application disclose a method for processing a 3D image, the method comprising: obtaining at least two slices in a direction of the 3D image from the 3D image; processing the at least two slices in the direction using a transformation model to obtain the at least two processed slices in the direction, and determining a transformed 3D image in the direction based on the at least two processed slices.

[0021] On the other hand, some embodiments of the present application disclose a non-transitory readable medium, the medium comprising at least one set of instructions for processing a 3D image. When at least one processor executes the at least one set of instructions, the at least one processor executes a method, the method comprising: obtaining at least two slices in a direction of the 3D image from the 3D image; processing the at least two slices in the direction using a transformation model to obtain at least two processed slices in the direction, and determining a transformed 3D image in the direction based on the at least two processed slices.

[0022] On the other hand, some embodiments of the present application disclose a system, including: at least one storage device, the at least one storage device including a set of instructions; and at least one processor communicating with the at least one storage device, when executing the set of instructions, the at least one processor is configured to cause the system to perform operations, the operations including: identifying at least two second slices corresponding to at least two converted 3D images in different directions, the at least two converted 3D images being obtained by processing first slices of the different directions of the 3D image; and sending the at least two second slices corresponding to the different directions to at least one terminal for display.

[0023] In some embodiments, the operation further includes: generating and sending a notification to the at least one terminal, wherein the at least two second slices are obtained from the at least two converted 3D images, wherein each of the converted 3D images is obtained by processing a slice in one of the different directions.

[0024] In some embodiments, the at least two transformed 3D images are acquired by: processing the slices of the different directions using at least two transformation models, each of the at least two transformation models corresponding to one of the different directions; and determining each of the at least two transformed 3D images based on the at least two processed slices of the corresponding directions.

[0025] In some embodiments, the at least two transformation models corresponding to the at least two directions are different from each other.

[0026] In some embodiments, the at least two transformed 3D images are acquired by: processing the slices at the different directions using a transformation model; and determining each of the at least two transformed 3D images based on the at least two processed slices at the corresponding directions.

[0027] In some embodiments, the visualization of the 3D image along multiple view planes in different directions is generated by synchronously displaying a second slice of the transformed 3D image along corresponding view planes, wherein the multiple view planes in the multiple directions intersect at a selected point in the 3D image. The transformed 3D image is processed along the corresponding multiple view planes in the same or similar directions.

[0028] In some embodiments, the selected point is interactively selected by a user, or automatically determined based on a predetermined trajectory.

[0029] Some embodiments of the present application also disclose an image processing system, comprising: at least one storage device, the at least one storage device comprising a set of instructions; and at least one processor communicating with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to perform operations, the operations comprising: acquiring a megavoltage CBCT image; processing slices of the megavoltage CBCT image using a transformation model to determine a kilovoltage MSCT image; and determining a region of interest (ROI) of the kilovoltage MSCT image.

[0030] In some embodiments, the operation further comprises: acquiring a treatment plan, the treatment plan being determined based on the ROI of the kilovoltage MSCT image.

[0031] In some embodiments, the operation further includes: determining a registration result by registering the kilovoltage MSCT image with the planning image; and determining whether to modify the treatment plan according to the registration result.

[0032] Some additional features of the present application may be described in the following description. Some additional features of the present application will be apparent to those skilled in the art through study of the following description and corresponding drawings or understanding of the production or operation of the embodiments. The features of the present disclosure may be realized by practicing or using various aspects of the methods, tools, and combinations set forth in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present application will be further described by exemplary embodiments. These exemplary embodiments will be described in detail by the accompanying drawings. The drawings are not drawn to scale. These embodiments are non-limiting exemplary embodiments, in which the same numbers in the various figures represent similar structures, wherein:

[0034] Figure 1 is a schematic diagram of an exemplary image processing system according to some embodiments of the present application;

[0035] Figure 2 is a schematic diagram of hardware and / or software components of an exemplary computing device that can implement a processing device according to some embodiments of the present application;

[0036] Figure 3 is a schematic diagram of hardware and / or software components of an exemplary mobile device according to some embodiments of the present application;

[0037] Figure 4A is a framework diagram of an exemplary processing device according to some embodiments of the present application;

[0038] Figure 4B is a framework diagram of an exemplary processing device according to some embodiments of the present application;

[0039] Figure 5 is an exemplary flow chart for processing a 3D image of a target according to some embodiments of the present application;

[0040] Figure 6 is an exemplary flow chart for generating a conversion model in one direction according to some embodiments of the present application;

[0041] Figure 7 is an exemplary flow chart of simulation operations according to some embodiments of the present application;

[0042] Figure 8 is a schematic diagram of an axial MSCT slice according to an exemplary test shown in some embodiments of the present application;

[0043] Fig. 9 is a schematic diagram of a sagittal MSCT slice according to an exemplary test shown in some embodiments of the present application;

[0044] Fig.10 is a schematic diagram of a coronal MSCT slice according to an exemplary test shown in some embodiments of the present application;

[0045] Fig.11 is a schematic diagram of an axial CBCT slice of an exemplary test according to some embodiments of the present application;

[0046] Fig.12 is a schematic diagram of a sagittal CBCT slice of an exemplary test according to some embodiments of the present application;

[0047] Fig.13 is a schematic diagram of a coronal CBCT slice according to an exemplary test shown in some embodiments of the present application;

[0048] Fig.14 is a schematic diagram of an exemplary axial slice of an axially transformed 3D CBCT image according to some embodiments of the present application;

[0049] Fig.15 is a schematic diagram of an exemplary sagittal slice of an axially transformed 3D CBCT image according to some embodiments of the present application;

[0050] Fig.16 is a schematic diagram of an exemplary coronal slice of an axially transformed 3D CBCT image according to some embodiments of the present application;

[0051] Fig.17 is a schematic diagram of an axial CBCT slice according to an exemplary process shown in some embodiments of the present application;

[0052] Fig.18 is a schematic diagram of an exemplary processed sagittal CBCT slice according to some embodiments of the present application; and

[0053] Fig.19 is a schematic diagram of an exemplary processed coronal CBCT slice according to some embodiments of the present application. DETAILED DESCRIPTION

[0054] The following description can enable those skilled in the art to make and use the present disclosure, and provide requirements in the context of specific applications. For those of ordinary skill in the art, it is obvious that various changes can be made to the disclosed embodiments, and the general principles defined in the present application can be applied to other embodiments and application scenarios without departing from the principles and scope of the present application. Therefore, the present disclosure is not limited to the embodiments shown, but conforms to the widest scope consistent with the claims.

[0055] The terms used in this application are only for the purpose of describing specific example embodiments and are not restrictive. As used in this application, the singular forms "one", "an", and "the" may also include plural forms unless the context clearly indicates an exception. It should also be understood that the term "comprising" when used in this disclosure specifies the presence of representative characteristics, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other characteristics, integers, steps, operations, elements, components, and / or groups, etc.

[0056] Generally, the words "module", "unit" or "block" used herein refer to logic embodied in hardware or firmware, or a collection of software instructions. The modules, units or blocks described herein may be implemented as software and / or hardware, and may be stored in any type of non-temporary computer-readable medium or other storage device. In some embodiments, software modules / units / blocks may be compiled and linked to an executable program. It should be understood that software modules may be called from other modules / units / blocks or themselves, and / or may be called in response to detected events or interruptions. Software modules / units / blocks configured for execution on a computing device may be provided on a computer-readable medium, such as a compact disc, a digital video disc, a flash drive, a disk, or any other tangible medium, or as a digital download (and may be initially stored in a compressed or installable format, requiring installation, decompression or decryption prior to execution). The software code here may be partially or completely stored in a storage device of a computing device performing the operation, and applied in the operation of the computing device. Software instructions may be embedded in firmware, such as an erasable programmable read-only memory (EPROM). It should also be understood that hardware modules / units / blocks may be included in connected logical components, such as gates and triggers, and / or may include programmable units, such as programmable gate arrays or processors. The modules / units / blocks or computing device functions described herein may be implemented as software modules / units / blocks, but may be represented in hardware or firmware. Generally, the modules / units / blocks described herein refer to logical modules / units / blocks, which may be combined with other modules / units / blocks or divided into sub-modules / sub-units / sub-blocks, although they are physical organizations or storage devices. This description may apply to a system, an engine, or a portion thereof.

[0057] It should be understood that the terms "system", "engine", "unit", "unit", "module", and / or "block" are methods for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, these terms may also be replaced by other expressions if the same purpose can be achieved.

[0058] It should be understood that when a unit, engine, module or block is referred to as being "on," "connected" or "coupled" to another unit, engine, module or block, it may be directly on, connected or coupled, or communicated to another unit, engine, module, or block, or an intermediate unit, engine, module, or block, unless the context clearly indicates otherwise. In this application, the term "and / or" may include any one or more of the relevant listed items or combinations thereof.

[0059] These and other features of the present disclosure and methods of operation and function of structures and structure-related elements and combinations of parts and economies of manufacture may become more apparent upon consideration of the following description with respect to the accompanying drawings. It should be understood, however, that the drawings are for illustration and description purposes only and are not intended to limit the scope of the present application. It should be understood that the drawings are not drawn to scale.

[0060] The flowcharts used for the present disclosure illustrate the operations implemented by the system according to some embodiments of the present disclosure. It should be understood that the operations in the flowcharts may not be performed in order. On the contrary, the operations may be implemented in an inverted order or simultaneously. At the same time, one or more other operations may also be added to these flowcharts. One or more operations may also be deleted from the flowcharts.

[0061] One aspect of the present disclosure relates to a system and method for processing 3D images. The system and method can apply a conversion model to a slice in one direction of a 3D image (e.g., corresponding to the direction of an axial plane, a coronal plane, or a sagittal plane of the 3D image), and obtain a converted 3D image of the direction based on the processed slice. The system and method can use different conversion models to process slices in different directions. In some embodiments, the model can be configured to generate a 3D image with improved image quality (in terms of desired display or task performance) by processing slices in a certain direction of a 3D image with lower image quality. In some embodiments, the conversion model can be configured to generate a 3D image of a second modality based on a 3D image of a first modality by processing slices in a specific direction of the 3D image of the first modality, wherein the first modality and the second modality are different. A 2D sample slice in the direction of a 3D training image can be used to train a conversion model corresponding to the direction. Compared with model training using 3D training images directly, model training using 2D sample slices as disclosed herein can converge more easily and consume less computing resources. The converted 3D image or one or more slices thereof may have improved image quality compared to the original 3D image or corresponding one or more slices, such as fewer artifacts, reduced noise, improved display, increased contrast, and / or include different or additional information (e.g., available or visible in the converted 3D image of the second modality but not available or visible in the original 3D image of the first modality), etc.

[0062] Figure 1 1 is a schematic diagram of an exemplary image processing system 100 according to some embodiments of the present application. The imaging system 100 may include an image acquisition device 110, a processing device 120, a storage device 130, one or more terminals 140, and a network 150. The components in the imaging system 100 may be connected in one or more ways. As an example only, the image acquisition device 110 may be connected to the processing device 120 via the network 150. As another example, the image acquisition device 110 may be directly connected to the processing device 120, such as Figure 1 As another example, the terminal 140 may be connected to another component of the imaging system 100 (e.g., the processing device 120) via the network 150. As yet another example, the terminal 140 may be directly connected to the processing device 120, such as Figure 1 As another example, the storage device 130 may be directly connected to another component of the imaging system 100 (eg, the processing device 120), such as Figure 1 shown.

[0063] The image acquisition device 110 may be configured to acquire image data of a target. The image data of a target may include an image (e.g., an image slice), projection data, etc., or a combination thereof. In some embodiments, the image data may be two-dimensional (2D) image data, three-dimensional (3D) image data, four-dimensional (4D) image data (e.g., a series of 3D image data over time), etc., or any combination thereof. The target may be biological or non-biological. For example, the target may include a person, an artificial object, etc., or a part thereof. In some embodiments, the image acquisition device 110 may include a medical imaging device 111, a camera 112, etc. In some embodiments, the medical imaging device 111 may include a computed tomography (CT) device (e.g., a cone beam CT (CBCT) device, a fan beam CT (FBCT) device, a multi-slice CT (MSCT) device, etc.), a magnetic resonance imaging (MRI) device, an ultrasound imaging device, a fluorescence imaging device, a single photon emission computed tomography (SPECT) device, a positron emission tomography (PET) device, an X-ray imaging device, etc., or any combination thereof.

[0064] The processing device 120 may process data and / or information obtained from the image acquisition device 110, the terminal 140, and / or the storage device 130. For example, the processing device 120 may obtain 2D image data of the target from the image acquisition device 110, and reconstruct the 2D image data to obtain a 3D image of the target. As another example, the processing device 120 may obtain at least two slices from a 3D image along a direction. The processing device 120 may apply a conversion model on the at least two slices to obtain at least two processed slices. The processing device 120 may also determine a converted 3D image of the direction based on the at least two processed slices. The processing device 120 may send at least one second slice of the converted 3D image to the terminal 140 for display on a display of the terminal 140. In some embodiments, the processing device 120 may be a computer, a user console, a single server or a server group, etc. The server group may be centralized or distributed. In some embodiments, the processing device 120 may be local or remote. For example, the processing device 120 may access information and / or data stored in the image acquisition device 110, the terminal 140, and / or the storage device 130 via the network 150. As another example, the processing device 120 may be directly connected to the image acquisition device 110, the terminal 140, and / or the storage device 130 to access the stored information and / or data. In some embodiments, the processing device 120 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof. In some embodiments, the processing device 120 may be provided by a server or a computer. Figure 2 The illustrated computing device 200 has one or more components to implement.

[0065] The storage device 130 may store data, instructions, and / or any other information. In some embodiments, the storage device 130 may store data obtained from the image acquisition device 110, the terminal 140, and / or the processing device 120. For example, the storage device 130 may store image data acquired by the image acquisition device 110. As another example, the storage device 130 may store conversion models corresponding to various directions. In some embodiments, the storage device 130 may store data and / or instructions, that is, the processing device 120 may execute or be used to execute the exemplary method / system described in the present disclosure. In some embodiments, the storage device 130 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), etc., or any combination thereof. Exemplary mass storage may include magnetic disks, optical disks, solid-state drives, etc. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compressed disks, tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM), and zero capacitance random access memory (Z-RAM), etc. Exemplary ROM may include mask ROM (MROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), compact disc ROM (CD-ROM), and digital versatile disk, etc. In some embodiments, the storage device 130 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.

[0066] In some embodiments, the storage device 130 may be connected to the network 150 to communicate with one or more other components in the imaging system 100 (e.g., the processing device 120, the terminal 140, etc.). One or more components in the imaging system 100 may access data or instructions stored in the storage device 130 through the network 150. In some embodiments, the storage device 130 may be directly connected to one or more other components in the imaging system 100 (e.g., the processing device 120, the terminal 140, etc.). In some embodiments, the storage device 130 may be part of the processing device 120.

[0067] The terminal 140 may enable user interaction between the user and the imaging system 100. In some embodiments, the terminal 140 may be connected and / or communicated with the image acquisition device 110, the processing device 120, the terminal 140, and / or the storage device 130. For example, the terminal 140 may display an image (e.g., a 3D image, a slice of a 3D image, etc.) of a target acquired from the processing device 120 or the image acquisition device 110. The terminal 140 may include a mobile device 141, a tablet computer 142, a laptop computer 143, etc., or any combination thereof. In some embodiments, the mobile device 141 may include a smart home device, a wearable device, a mobile device, a virtual reality device, an augmented reality device, etc., or any combination thereof. In some embodiments, the smart home device may include a smart lighting device, a smart appliance control device, a smart monitoring device, a smart TV, a smart camera, an intercom, etc., or any combination thereof. In some embodiments, the wearable device may include a bracelet, a shoe, a pair of glasses, a helmet, a watch, a piece of clothing, a backpack, a smart accessory, etc., or any combination thereof. In some embodiments, the mobile device may include a mobile phone, a personal digital assistant (PDA), a gaming device, a navigation device, a point of sale (POS) device, a laptop, a tablet computer, a desktop computer, etc., or any combination thereof. In some embodiments, the virtual reality device and / or the augmented reality device may include a virtual reality helmet, virtual reality glasses, virtual reality goggles, an augmented reality helmet, augmented reality glasses, augmented reality goggles, etc., or any combination thereof. For example, the virtual reality device and / or the augmented reality device may include Google Glass TM , Oculus Rift TM , aHololens TM , Gear VR TM Etc. In some embodiments, the terminal 140 may be a part of the processing device 120 .

[0068] The network 150 may include any suitable network that can facilitate the exchange of information and / or data for the imaging system 100. In some embodiments, one or more components of the image acquisition device 110 (e.g., an MRI device), the terminal 140, the processing device 120, the storage device 130, etc. may communicate information and / or data with one or more other components of the imaging system 100 via the network 150. For example, the processing device 120 may obtain data from the image acquisition device 110 via the network 150. As another example, the processing device 120 may obtain user instructions from the terminal 140 via the network 150. The network 150 may and / or include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN), a wide area network (WAN), etc.), a wired network (e.g., Ethernet), a wireless network (e.g., an 802.11 network, a Wi-Fi network, etc.), a cellular network (e.g., a long term evolution (LTE) network), a frame relay network, a virtual private network (“VPN”), a satellite network, a telephone network, a router, a hub, a switch, a server computer, etc., or any combination thereof. By way of example only, the network 150 may include a cable network, a wired network, a fiber optic network, a telecommunication network, an intranet, a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth TM Network, ZigBee TM In some embodiments, the network 150 may include one or more network access points. For example, the network 150 may include a wired and / or wireless network access point, such as a base station and / or an Internet exchange point, and one or more components of the imaging system 100 may be connected to the network 150 to exchange data and / or information.

[0069] It should be noted that the above description of the imaging system 100 is for illustration only and is not intended to limit the scope of the present disclosure. For those skilled in the art, various changes and modifications can be made based on the description of the present application. For example, the components and / or functions of the imaging system 100 can be changed or altered according to the specific implementation scenario.

[0070] Figure 2 is a schematic diagram of hardware and / or software components of an exemplary computing device 200 according to some embodiments of the present application. The computing device 200 can be used to implement any component of the imaging system 100 as described herein. For example, the processing device 120 and / or the terminal 140 can be implemented on the computing device 200 through its hardware, software program, firmware, or a combination thereof. Although only one such computing device is shown, for convenience, computer functions related to similar imaging systems 100 as described herein can be implemented in a distributed manner on multiple similar platforms to distribute the processing load. Figure 2 As shown, computing device 200 may include processor 210 , memory 220 , input / output (I / O) 230 , and communication port 240 .

[0071] The processor 210 can execute computer instructions (program code) according to the techniques described herein and perform the same functions as the processing device 120. Computer instructions may include, for example, routines, programs, objects, components, signals, data structures, processes, modules, and functions that perform specific functions described herein. For example, the processor 210 can process data obtained from the image acquisition device 110, the terminal 140, the storage device 130, and / or any other component of the imaging system 100. Specifically, the processor 210 can process data obtained from the image acquisition device 110. For example, the processor 210 can generate an image (e.g., a 3D image, a 2D image, etc.) based on the data. In some embodiments, the generated image can be stored in the storage device 130, the memory 220, etc. In some embodiments, the generated image can be displayed on a display device through the input / output 230. In some embodiments, the processor 210 can execute instructions obtained from the terminal 140. In some embodiments, the processor 210 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), etc. Any circuit or processor capable of performing one or more functions, or any combination thereof.

[0072] For illustration purposes only, only one processor is described in the computing device 200. However, it should be noted that the computing device 200 in the present disclosure may also include multiple processors. Therefore, the operations and / or method steps performed by one processor as described in the present disclosure may also be performed jointly or individually by multiple processors. For example, if in the present disclosure, the processor of the computing device 200 performs operation A and operation B, it should be understood that operation A and operation B may also be performed jointly or individually by two or more different processors in the computing device 200 (e.g., the first processor performs operation A and the second processor performs operation B, or the first processor and the second processor jointly perform operation A and operation B).

[0073] The memory 220 may store data / information obtained from the image acquisition device 110, the terminal 140, the storage device 130, or any other component of the imaging system 100. In some embodiments, the memory 220 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. For example, the mass storage may include a magnetic disk, an optical disk, a solid-state drive, etc. The removable storage device may include a flash drive, a floppy disk, an optical disk, a memory card, a zipper disk, a magnetic tape, etc. The volatile read-write memory may include a random access memory (RAM). The RAM may include a dynamic RAM (DRAM), a dual date rate synchronous dynamic RAM (DDRSDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and a zero capacitance RAM (Z-RAM), etc. The ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erasable programmable ROM (PEROM), an erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digital multifunction ROM, etc. In some embodiments, the memory 220 may store one or more programs and / or instructions to perform the exemplary methods described in the present disclosure.

[0074] Input / output 230 can input or output signals, data and / or information. In some embodiments, input / output 230 can enable a user to interact with processing device 120. In some embodiments, input / output 230 can include input devices and output devices. Exemplary input devices can include a keyboard, a mouse, a touch screen, a microphone, etc., or any combination thereof. Exemplary output devices can include a display device, a speaker, a printer, a projector, etc., or any combination thereof. Exemplary display devices can include a liquid crystal display (LCD), a display based on a light emitting diode (LED), a flat panel display, a curved screen, a television set, a cathode ray tube (CRT), etc., or any combination thereof.

[0075] The communication port 240 can be connected to a network (e.g., network 150) for data communication. The communication port 240 can establish a connection between the processing device 120 and the image acquisition device 110, the terminal 140, or the storage device 130. The connection can be a wired connection, a wireless connection, or a combination of the two, so that data can be transmitted and received. The wired connection can include a cable, an optical cable, a telephone line, etc., or any combination thereof. The wireless connection can include a Bluetooth network, a Wi-Fi network, a WiMAX network, a WLAN, a ZigBee network, a mobile network (e.g., 3G, 4G, 5G, etc.), etc., or any combination thereof. In some embodiments, the communication port 240 can be a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port 240 can be a specially designed communication port. For example, the communication port 240 can be designed according to the digital imaging and communication in the medical (DICOM) protocol.

[0076] Figure 3 FIG. 3 is a schematic diagram of hardware and / or software components of an exemplary mobile device 300 according to some embodiments of the present application. In some embodiments, one or more terminals 140 and / or processing devices 120 may be implemented on the mobile device 300, respectively.

[0077] like Figure 3 As shown, the mobile device 300 may include a communication platform 310, a display screen 320, a graphics processing unit (GPU) 330, a central processing unit (CPU) 340, an input / output (I / O) 350, a memory 360, and a storage 390. In some embodiments, any other suitable components, including but not limited to a system bus or a controller (not shown), may also be included in the mobile device 300. In some embodiments, a mobile operating system (OS) 370 (e.g., iOS, Android, Windows Phone, etc.) and one or more applications 380 may be loaded from the storage 390 into the memory 360, which is executed by the CPU 340. The applications (APPs) 380 may include a browser or any other suitable mobile application for receiving and presenting information related to image data acquisition or other information from the processing device 120. User interaction with the information stream may be achieved through the input / output 350 and provided to the processing device 120 and / or other components of the imaging system 100 via the network 150.

[0078] In order to implement the various modules, units and functions described in the present disclosure, a computer hardware platform can be used as a hardware platform for one or more elements described herein. The hardware elements, operating systems and programming languages ​​of such computers are traditional and are considered to be sufficiently familiar to those skilled in the art to adapt to those image data acquisition techniques as described herein. A computer with user interface elements can be used to implement a personal computer (PC) or other type of workstation or terminal device, although the computer can also act as a server if it is appropriately programmed. It is believed that those skilled in the art are familiar with the structure, programming and general operation of such computer equipment, and the accompanying drawings can illustrate this.

[0079] Figure 4A and Figure 4B is a schematic diagram of exemplary processing devices 120A and 120B according to some embodiments of the present application. In some embodiments, processing devices 120A and 120B may be Figure 1 In some embodiments, the processing devices 120A and 120B may be respectively in a processing unit (e.g., Figure 2 The processor 210 shown in Figure 3 340 shown in FIG. 3). As an example only, processing device 120A may be implemented on CPU 340 of a terminal device, and processing device 120B may be implemented on computing device 200. Alternatively, processing devices 120A and 120B may be implemented on the same computing device 200 or the same CPU 340. For example, processing devices 120A and 120B may be implemented on the same computing device 200.

[0080] like Figure 4A As shown, the processing device 120A may include a slice acquisition module 410 , a processing module 420 , a reconstruction module 430 , a display module 440 and a model acquisition module 450 .

[0081] The slice acquisition module 410 may be configured to acquire a slice (or referred to as a first slice) of a 3D image. For example, the slice acquisition module 410 may acquire at least two slices in one direction from the 3D image. In some embodiments, the direction may be any direction in space. The direction may be defined relative to the 3D image. For example, the direction may be related to an axial plane, a coronal plane, a sagittal plane, etc. of a target in the 3D image. For example, the direction may be a direction parallel to an axial plane, a coronal plane, a sagittal plane, etc. of the 3D image. In some embodiments, at least two slices may be 2D images of the 3D image. In some embodiments, the slice acquisition module 410 may retrieve at least two slices along a direction from the 3D image. For example, the slice acquisition module 410 may identify a plurality of planes and a transformation matrix of each of the plurality of planes along a direction of the 3D image. One of the at least two slices may be acquired based on an array of 3D images and a transformation matrix of each of the at least two planes. As another example, the slice acquisition module 410 may acquire at least two first slices in a second direction from the 3D image. In some embodiments, the first direction may be different from the second direction.

[0082] The processing module 420 may be configured to process the slices (or referred to as first slices) obtained from the slice acquisition module 410. For example, the processing module 420 may process at least two slices in the direction using a conversion model. In some embodiments, the conversion model may be a process or algorithm for processing a 2D image. For example, the conversion model may be a deep learning model. In some embodiments, the processing module 420 may input at least two slices in the direction into the conversion model. The output of the conversion model may include at least two processed slices in the direction, each processed slice corresponding to one of the at least two input slices in the direction. As another example, the processing module 420 may process at least two second slices in the second direction using a second conversion model to obtain at least two second processed slices in the second direction. In some embodiments, the second conversion model may be different from the first conversion model.

[0083] The reconstruction module 430 may be configured to reconstruct the transformed 3D image of the direction. For example, the reconstruction module 430 may determine the transformed 3D image in the direction based on at least two processed slices in the direction. In some embodiments, the reconstruction module 430 may align the at least two processed slices in the direction and stack the aligned slices to obtain the transformed 3D image. As another example, the reconstruction module 430 may determine a second transformed 3D image in the second direction based on at least two second processed slices in the second direction.

[0084] The display module 440 may be configured to obtain a slice (also referred to as a first slice) of a 3D image (original 3D image) to be displayed and / or a second slice of a converted 3D image. For example, the display module 440 may obtain at least one second slice in the first direction to be displayed from the converted 3D image of the first direction. In some embodiments, the display module 440 may obtain at least one second slice from the converted 3D image based on a user instruction. For example, a user may select at least one second slice to be displayed via the user terminal 140. As another example, a user may select an object of a target represented in the converted 3D image (e.g., an organ of a target in the converted 3D image) via the user terminal 140, and the display module 440 may identify one or more slices including a description of the target as at least one second slice. In some embodiments, the display module 440 may send at least one second slice to the terminal 140. The terminal 140 may display at least one second slice on a user interface (e.g., a display screen) of the terminal 140. As another example, the display module 440 may obtain at least one second slice in the second direction to be displayed from the second converted 3D image of the second direction.

[0085] The model acquisition module 450 may be configured to acquire a training model. For example, the model acquisition module 450 may acquire a conversion model of a first direction. As another example, the model acquisition module 450 may acquire a second conversion model of a second direction.

[0086] like Figure 4B As shown, the processing device 120B may include a training sample acquisition module 460 , a slice acquisition module 470 , a training module 480 , and a sample synthesis module 490 .

[0087] The training sample acquisition module 460 may be configured to acquire training samples. For example, the training sample acquisition module 460 may acquire a plurality of training samples (or referred to as sample image pairs) for training the conversion model (or the second conversion model). The sample image pairs may include a sample target image and a corresponding sample input image (e.g., a sample MSCT image as a sample target image and a sample CBCT image as a sample input image) for model training. The sample image pairs may include two corresponding images of different image qualities and / or different modalities. As used herein, if each image includes images representing the same target or the same portion thereof, the images are considered to correspond to each other.

[0088] The slice acquisition module 470 may be configured to obtain sample slices of the training sample. For example, the slice acquisition module 470 may acquire at least two sample slices from the sample image (e.g., the sample target image, the sample input image) along the direction. For example, the slice acquisition module 470 may identify at least two planes along the direction of the sample image and the transformation matrix of each of the at least two planes. One of the at least two sample slices may be acquired based on the array of the sample image and the transformation matrix of each of the at least two planes. Each of at least a portion of the sample slices of the sample input image may correspond to a sample slice of the sample target image.

[0089] The training module 480 may be configured to train a preliminary model to obtain a conversion model (or a second conversion model). For example, the training module 480 may train a preliminary model using at least two sample slices in the direction (or second direction) of the sample input image in each of the plurality of training samples and corresponding sample slices in the direction (or second direction) of the target image to obtain a conversion model (or a second conversion model). In some embodiments, a conversion model in a first direction may be trained using sample slices in a first direction. In some embodiments, a conversion model in a first direction may be trained using sample slices in one or more second directions different from the first direction. In some embodiments, the training module 480 may train a preliminary model using sample slices in at least two different directions to obtain a single conversion model. A single conversion model may be used to process slices in any one of at least two different directions. For example, a single conversion model may be trained using sample slices in a coronal plane, sample slices in an axial plane, and sample slices in a sagittal plane, and may be used to process slices in any of the axial plane, sagittal plane, and coronal plane, or slices in any other direction.

[0090] The sample synthesis module 490 may be configured to synthesize the sample input image based on the sample target image. For example, the sample synthesis module 490 may convert the sample target image into at least one 3D array, and obtain projection values ​​of the sample target image by performing a projection algorithm and a noise simulation algorithm on the sample target image based at least in part on the at least one 3D array. The sample synthesis module 490 may determine the synthesized sample image by reconstructing the projection values ​​using a reconstruction algorithm.

[0091] It should be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. Obviously, for those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications do not depart from the scope of the present application. For example, the processing module 420 and the reconstruction module 430 can be integrated into a single module. As another example, some other components / modules (e.g., storage modules) can be added and / or omitted in the processing devices 120A and 120B. As another example, the processing device 120A and the processing device 120B can be integrated into one processing device 120. In some embodiments, the processing device 120A and / or the processing device 120B can share two or more modules, and any module can be divided into two or more units. For example, the processing devices 120A and 120B can share the same slice acquisition module; that is, the slice acquisition module 410 and the slice acquisition module 470 are the same module. Each of the above modules can be a hardware circuit designed to perform certain actions, for example, according to instructions stored in one or more storage media, and / or any combination of a hardware circuit and one or more storage media.

[0092] Figure 5 is an exemplary flowchart for processing a 3D image of a target according to some embodiments of the present application. In some embodiments, process 500 can be implemented as a set of instructions (e.g., an application) stored in storage device 130, memory 220, or memory 390. Processing device 120, processor 210, and / or CPU 340 can execute the instructions, and when executing the instructions, processing device 120, processor 210, and / or CPU 340 can be configured to execute process 500. The operations of the process shown below are for illustrative purposes only. In some embodiments, process 500 can be completed with one or more additional operations not described and / or one or more operations not discussed. In addition, Figure 5 The order in which flow 500 is shown and the operations described below are not intended to be limiting.

[0093] In step 502 , the processing device 120A (eg, the slice acquisition module 410 ) may acquire at least two slices (or referred to as first slices) in one direction from a 3D image.

[0094] In some embodiments, the 3D image may be directly obtained from the image acquisition device 110. For example, the image acquisition device 110 may include a 3D imaging device to capture a 3D image of the target and send the 3D image to the processing device 120A. In some embodiments, the processing device 120A may acquire image data of the target from the image acquisition device 110. The image data may be reconstructed to acquire the 3D image. In some embodiments, the 3D image may be a 3D medical image. For example, the 3D image may include a CBCT image, an FBCT image, an MSCT image, an MRI image, an ultrasound image, a SPECT image, a PET image, an X-ray image, etc., or any combination thereof.

[0095] In some embodiments, the direction can be any direction in space. The direction can be defined relative to the 3D image. For example, the direction can be related to the axial plane, coronal plane, sagittal plane, etc., in which the target is represented in the 3D image. For example, the direction can be a direction parallel to the axial plane, coronal plane, sagittal plane, etc. of the 3D image. In some embodiments, the axial plane can be an imaginary plane that divides the target (e.g., a human body) into an upper part and a lower part. In some embodiments, the coronal plane can be an imaginary plane that divides the target into a front part and a back part. For example, the target is a human body, and the coronal plane can divide the human body into an abdominal part and a back part. In some embodiments, the sagittal plane can be an imaginary plane that divides the target (e.g., a human body) into a right part and a left part.

[0096] In some embodiments, at least two slices (or referred to as first slices) may be 2D images of a 3D image. In some embodiments, the processing device 120A may acquire at least two slices from the 3D image along the direction. For example, the processing device 120A may identify at least two planes along the direction of the 3D image and the transformation matrix of each of the at least two planes. Based on the transformation matrix of each of the at least two planes and the array of the 3D image, one of the at least two slices may be obtained. In some embodiments, the processing device 120A may identify a target in the 3D image (e.g., tissue of a human body in the 3D image). The processing device 120A may identify one or more slices including the target as the at least two slices.

[0097] In step 504 , the processing device 120A (eg, the processing module 420 ) may process at least two slices (or referred to as first slices) in the direction using the transformation model.

[0098] In some embodiments, the conversion model can be a process or algorithm for processing 2D images. In some embodiments, the conversion model can be an image style transfer model. The image style transfer model can convert an image from a first type from a first category to a second type (e.g., from a thermal image of an image to an RGB image of an image). In some embodiments, the conversion model can be a deep learning model. For example, the conversion model can include a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generated adversary network (GaN) model, a Cycleangan model, a PIX2PIX model, etc., or any combination.

[0099] In some embodiments, the conversion model may be related to direction. The conversion model may be trained using training samples including 2D images (or slices) in a direction. For example, the direction may be parallel to the coronal plane of the 3D image. The conversion model may be trained using multiple training samples. The training samples may include 3D image pairs corresponding to the same sample target. Each of the 3D image pairs may include at least two 2D images in a direction, such as at least two 2D images in the coronal plane. In some embodiments, the conversion model may be related to at least two different directions. The conversion model may be trained using training samples including 2D images (or slices) in at least two different directions. For example, the training samples may include 2D images in different directions in the 3D image, such as at least two 2D images in the coronal plane, at least two 2D images in the axial plane, and at least two 2D images in the sagittal plane. As another example, a single conversion model may be obtained using sample slices in at least two different directions. A single conversion model can be used to process slices in any of at least two different directions. More descriptions on the training of conversion models can be found elsewhere in this disclosure. See. Figure 6 and its related description.

[0100] In some embodiments, the processing device 120A may input the at least two slices of the direction into the transformation model. The output of the transformation model may include at least two processed slices in the direction, each processed slice corresponding to one of the at least two input slices in the direction.

[0101] In step 506 , the processing device 120A (eg, the reconstruction module 430 ) may determine a transformed 3D image of the direction based on the at least two processed slices of the direction.

[0102] In some embodiments, the processing device 120A may align at least two processed slices in the direction and stack the aligned slices to obtain a converted 3D image. In some embodiments, the processing device 120A may reconstruct the converted 3D image based on at least two processed slices in the direction. Exemplary reconstruction algorithms may include a point technique based on stereo correspondence, a non-stereo corresponding contour method (NCS), a surface rendering technique, a projective reconstruction technique, an affine reconstruction technique, a Euclidean reconstruction technique, etc., or any combination thereof.

[0103] In step 508 , the processing device 120A (eg, the display module 440 ) may acquire at least one second slice for display based on the at least two processed slices or from the directional converted 3D image.

[0104] In some embodiments, the processing device 120A may obtain one or more second slices for display from at least two processed slices of the converted 3D image of the direction. That is, the second slice for display is the same as the processed slice. In some embodiments, the processing device 120A may generate the second slice based on obtaining one or more processed slices from at least two processed slices of the converted 3D image of the direction. For example, the processing device 120A may generate the second slice based on one or more of the at least two processed slices by interpolation, extrapolation, etc.

[0105] In some embodiments, the processing device 120A may acquire at least one second slice from the converted 3D image based on a user instruction. For example, the user may select at least one second slice to be displayed via the user terminal 140. As another example, the user may select an object of a target represented in the converted 3D image (e.g., an organ of a target in the converted 3D image) via the user terminal 140, and the processing device 120A may identify at least one second slice including the display of the target as one or more second slices. In some embodiments, the processing device 120A may send the at least one second slice to the terminal 140. The terminal 140 may display the at least one second slice on a user interface (e.g., a display screen) of the terminal 140.

[0106] In some embodiments, the processing device 120A may process the (original) 3D image along a second direction different from the direction described in steps 502-506 of process 500. For example, for the second direction, the processing device 120A may obtain at least two slices in the second direction from the 3D image. For example, the direction may be parallel to the sagittal plane, and the second direction may be parallel to the coronal plane of the 3D image. The processing device 120A may process the at least two slices in the second direction by a second transformation model. In some embodiments, the second transformation model may be different from the transformation model described in step 504. For example, the transformation model in step 502 may be trained with sample slices in the direction (e.g., sample slices in the sagittal plane), and the second transformation model may be trained with sample slices in the second direction (e.g., sample slices in the coronal plane). In some embodiments, the processing device 120A may process at least two slices in the second direction (or referred to as the first slice) by the transformation model described in step 504 of process 500. For example, at least two slices in the coronal plane (e.g., the second direction) may be processed by a transformation model trained with sample slices in the sagittal plane (e.g., the direction).

[0107] To process the (original) 3D model along the second direction, at least two slices in the second direction may be input into the second transformation model, and the output of the second transformation model may be at least two processed slices in the second direction. The processing device 120A may determine a transformed 3D image in the second direction based on the at least two processed slices in the second direction. In some embodiments, the processing device 120A may further obtain at least one second slice in the second direction for display from the transformed 3D image in the second direction.

[0108] In some embodiments, the processing device 120A may identify at least two different directions relative to the 3D image. For each of the at least two directions, the processing device 120A may perform steps 502-508 of process 500. For example, the processing device 120A may identify three directions, each direction being parallel to an axial plane (a first direction), a coronal plane (a second direction), and an (original) sagittal plane (a third direction). The processing device 120A may perform process 500 for each of the three directions to obtain a transformed 3D image in the first direction (e.g., in the axial plane), a transformed 3D image in the second direction (e.g., in the coronal plane), and a transformed 3D image in the third direction (e.g., in the sagittal plane).

[0109] The processing device 120A may obtain the converted 3D image or part thereof from, for example, a storage device, and transmit it to one or more terminals 140 for display. In some embodiments, the received content may be displayed on multiple terminals 140 simultaneously and synchronously. Changes in the displayed content on one terminal 140 may also be reflected on another terminal 140. For example, a user may provide an instruction to zoom in or out on one terminal 140, and the zoom level of the displayed content on multiple terminals 140 may be changed accordingly. In some embodiments, a user may be allowed to specify changes displayed on one terminal 140, or synchronize the displayed content on multiple terminals 140.

[0110] Just referring to the example displayed on one terminal 140, two or more, namely at least one second slice in the direction, at least one second slice in the second direction, and at least one second slice in the third direction, can be displayed in different areas on the user interface (e.g., display screen) of the terminal 140. For example, the user interface may include a plurality of panels, in each of which at least one second slice in a direction may be displayed simultaneously and / or synchronously. In some embodiments, the processing device 120A may identify a second slice of the same object corresponding to the target in each of a plurality of converted 3D images in different directions. The processing device 120A may send the second slice for simultaneous and synchronous display to the terminal 140. For example, two (or three) second slices in two directions displayed simultaneously on the terminal 140. When the user turns on or provides an instruction pointing to a group of second slices in one direction, a group of second slices in another (or two other) directions may be changed in accordance with the user's operation. In some embodiments, at least one second slice in different directions may be displayed sequentially on the terminal 140. For example, the terminal 140 may display at least one second slice in the direction before at least one second slice in the second direction on the user interface (e.g., display screen) of the terminal 140. In some embodiments, the processing device 120A may generate a notification showing that different groups of second slices for display may be obtained from the converted 3D images in different directions. The converted 3D images in different directions may be obtained by processing slices in different directions of the same 3D image. The processing device 120A may also send the notification to the terminal 140. In some embodiments, the second slice and the notification may be displayed on the same terminal 140. In some embodiments, the second slice and the notification may be displayed on at least two different terminals 140. For example, the second slice may be displayed on the first terminal 140, and the notification may be a voice played on the second terminal 140 different from the first terminal 140.

[0111] In some embodiments, the viewing plane of the converted 3D image (or portion thereof) for display may be the same or similar to the direction of the viewing plane of at least two slices of the 3D image processed when the converted 3D image is obtained. For example, the difference between the viewing plane of the converted 3D image and the viewing plane of the corresponding slice processed when the converted 3D image is obtained along it may be lower than a predetermined angle threshold (e.g., 3 degrees, 5 degrees, 10 degrees, etc.). As another example, the converted 3D image may be obtained based on at least two axial slices of the 3D image. The viewing plane of the converted 3D image for display may be the axial plane of the converted 3D image. In some embodiments, the visualization of multiple viewing planes of the 3D image along different directions may be generated by synchronously displaying a second slice of the converted 3D image along the corresponding viewing plane, wherein the multiple viewing planes of the multiple directions may intersect at a selected point in the 3D image. The converted 3D image may be processed along the same or similar directions of the corresponding multiple viewing planes. For example, when a user wishes to synchronously view the 3D image along the axial plane, the coronal plane, and the sagittal plane, a point may be selected in the 3D image. The axial plane of the axially transformed 3D image, the coronal plane of the coronally transformed 3D image, and the sagittal plane of the sagittal transformed 3D image that intersect at the selected point can be determined. The axial second slice of the axial plane of the axially transformed 3D image, the coronal second slice of the coronal plane of the coronally transformed 3D image, and the sagittal second slice of the sagittal plane of the sagittal plane transformed 3D image can be displayed synchronously. In some embodiments, the selected point can be interactively selected by the user. In some embodiments, the selected point can be automatically determined based on a predetermined trajectory. For example, the predetermined trajectory can pass through the target (or 3D image) in an automatic manner. The selected point can be any point on the predetermined trajectory.

[0112] In some embodiments, the browsing process of browsing the converted 3D image corresponding to one or more directions through the 2D view can be basically the same as the browsing process of browsing the original 3D image (the converted 3D image is generated therefrom) via the 2D view. For example, by displaying the axial plane slices, coronal plane slices and sagittal plane slices of the 3D image, the original 3D image of the human body (or simply referred to as the 3D image) is shown on the user interface of the terminal 140. The three planes intersect at a selected point. During the browsing of the 3D image, the selected point can be moved along the axial direction (or along the intersection line of the coronal plane and the sagittal plane), so as to display the slices from the head of the human body to the toes along the axial direction as the selected point moves. During the browsing, the coronal plane slices and sagittal plane slices of the 3D image can remain the same. Similarly, the converted 3D image (obtained by processing the slices of the 3D image) can be browsed from one or more directions via the 2D view in the same way. For example, an axially transformed 3D image (obtained by processing the axial slices of the 3D image), a coronally transformed 3D image (obtained by processing the coronal slices of the 3D image), and a sagittal transformed 3D image (obtained by processing the sagittal slices of the 3D image) of the human body can be displayed on the user interface of the terminal 140 by displaying an axial second slice along the axial plane of the axially transformed 3D image, a coronal second slice along the coronal plane of the coronally transformed 3D image, and a sagittal second slice along the sagittal plane of the sagittal transformed 3D image. The axial plane, the coronal plane, and the sagittal plane can intersect at a selected point. During browsing of the transformed 3D image, the selected point can be moved along the axial direction of the axially transformed 3D image (or along the intersection of the coronal plane of the coronally transformed 3D image and the sagittal plane of the sagittal transformed 3D image), whereby a second slice from the head to the toes of the human body along the axial direction of the axially transformed 3D image can be displayed as the movement of the selected point. During browsing, the second slice of the coronal plane of the coronally transformed 3D image and the second slice of the sagittal plane of the sagittal transformed 3D image can remain the same. It should be noted that only describing the browsing process along the axial direction is only for illustrative purposes, and the 3D image and / or the transformed 3D image can be browsed in any known manner.

[0113] In some embodiments, the browsing process of obtaining a 2D view of a converted 3D image through a 2D operation as disclosed in the present invention may be the same as the browsing process of obtaining a converted 3D image through a 3D operation. As used in the present disclosure, a 3D operation of a 3D image refers to the conversion of a 3D image using a conversion model that operates directly on the 3D image, rather than operating on a 2D slice in one direction of the 3D image as described in the embodiments of the present disclosure. Such a conversion model used in the 3D operation can be directly trained with a 3D training image, rather than being obtained by training with a 2D slice in one direction of the 3D training image used in the embodiments of the present disclosure.

[0114] It should be noted that the above description is for illustration only and is not intended to limit the scope of the present disclosure. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications do not depart from the scope of the present application. In some embodiments, one or more operations can be omitted and / or one or more additional operations can be added. For example, one or more other optional operations (e.g., storage operations) can be added elsewhere in the process 500. In some embodiments, the processing device 120A can store the converted 3D image (e.g., the converted 3D image of the direction, the converted 3D image of the second direction, the converted 3D image of the third direction) or a part thereof (e.g., at least one second slice of the direction, at least one second slice of the second direction, at least one second slice of the third direction) in the storage device (e.g., storage device 130, memory 220, memory 390) of the imaging system 100.

[0115] Merely by way of example, a 3D C BCT image acquired by a CBCT device (image acquisition device 110) may be processed according to the method of the present disclosure. In terms of contrast, contrast-to-noise ratio, resolution, etc., a CBCT image may have poorer image quality than an MSCT image acquired by an MSCT device. For example, since the average atomic number of bones in an MSCT image (in turn, due to the calcium content in the bones) is higher than in a CBCT image, bones may be presented with higher contrast in an MSCT image. As another example, scattering and beam hardening of a CBCT device may reduce the image quality of a CBCT image.

[0116] In some embodiments, a megavoltage (MV) CBCT image may be processed according to the method of the present disclosure to obtain a kilovoltage (KV) MSCT image. For example, a conversion model may be trained using sample image pairs. Each sample image pair may include a sample MVCBCT image and a corresponding sample kilovoltage MSCT image. According to the method of the present disclosure, a MVCBCT image (or referred to as a 3D image) acquired by a CBCT device (image acquisition device 110) may be processed using a conversion model. For example, slices of an MVCBCT image may be acquired and processed using a conversion model to acquire processed slices. Based on the processed slices, a kilovoltage MSCT image (or referred to as a converted 3D image) corresponding to the MVCBCT image may be determined. In some embodiments, a kilovoltage MSCT image may be used to facilitate treatment of a target during a radiotherapy process (e.g., an adaptive radiation therapy (ART) process). For example, after acquiring a kilovoltage MSCT image, a region of interest (ROI) of the kilovoltage MSCT image may be determined by an ROI contour on the kilovoltage MSCT image. A treatment plan for treating a target may be determined based on the contour ROI on the kilovoltage MSCT image by a doctor or processing device 120A. In some embodiments, a planning image may be acquired before the radiotherapy process of the target. The planning image can be used to identify and locate one or more radiation targets of the target. The registration result can be determined by registering the kilovoltage MSCT image with the planning image. The registration result can be used to determine the difference between the kilovoltage MSCT image and the planning image, and to further optimize the treatment plan. For example, the processing device 120A can determine whether to modify the treatment plan based on the registration result. The improved treatment plan can be implemented on the target. In some embodiments, a DRR (digital reconstruction radiography) image can be generated by simulating the projection on the kilovoltage MSCT image. And a 2D image during radiotherapy can be acquired by a detector of a CBCT device (for example, an EPID of a radiotherapy device). Determining the registration result between the DRR image and the 2D image can also be used to determine whether to modify the treatment plan or adjust the machine parameters. Thus, the treatment plan can be optimized during radiotherapy.

[0117] In some embodiments, a conversion model can be trained to process slices of CBCT images to generate a converted 3D image similar to an MSCT image. For example, a preliminary model can be trained by at least two MSCT-CBCT image pairs to obtain a conversion model. The preliminary model can be a deep learning model. For example, the preliminary model can include a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generative adversarial network (GAN) model, a CycleanGan model, a PIX2PIX model, etc., or any combination thereof. In some embodiments, the image quality of CBCT images can be improved by training the conversion model, for example, by reducing scattering and beam hardening caused by the CBCT device, reducing image degradation noise, improving contrast, signal-to-noise ratio, resolution, etc., or any combination thereof. Exemplary processes for obtaining a conversion model can be found elsewhere (e.g., Figure 6-Figure 7 , and its related description).

[0118] In some embodiments, the processing device 120A can identify the direction of the CBCT image. The processing device 120A can obtain at least two slices in the direction from the CBCT image, and apply the transformation model of the direction to the at least two slices. For example, the input of the transformation model may include at least two slices, and the output of the transformation model may include at least two processed slices in the direction, each processed slice corresponding to one of the at least two slices. The processing device 120A can also generate a transformed CBCT image based on the at least two processed slices. One or more second slices of the transformed CBCT image of the direction can be displayed on the terminal 140.

[0119] In some embodiments, the processing device 120A may process the CBCT image using different transformation models in different directions. For example, for each direction parallel to the axial plane, the coronal plane, and the sagittal plane, the processing device 120A may acquire at least two axial slices, at least two coronal slices, and at least two sagittal slices. In some embodiments, the at least two axial slices, at least two coronal slices, and at least two sagittal slices may be input into the axial transformation model, the coronal transformation model, and the sagittal transformation model, respectively, to acquire at least two processed axial slices, at least two processed coronal slices, and at least two processed sagittal slices. Processing the at least two processed axial slices, the at least two processed coronal slices, and the at least two processed sagittal slices may generate a transformed axial CBCT image, a transformed coronal CBCT image, and a transformed sagittal CBCT image, respectively. In some embodiments, the processing device 120A may acquire at least one second axial slice of the axial transformed axial CBCT image for display, at least one second coronal slice of the transformed coronal CBCT image, and / or at least one second sagittal slice of the transformed sagittal CBCT image. For example, on the user interface of the terminal 140, the second axial slice, the second coronal slice, and the second sagittal slice may be displayed simultaneously in three separate panels. As another example, the terminal 140 may display only one or two of the second axial slice, the second coronal slice, and the second sagittal slice on the user interface at a time. Alternatively, all of the at least two axial slices, the at least two coronary slices, and the at least two sagittal slices may be input into the axial transformation model (or the coronary transformation model, or the sagittal transformation model) to obtain at least two processed axial slices, at least two processed coronal slices, and at least two processed sagittal slices.

[0120] In some embodiments, at least two sample axial MSCT-CBCT image pairs may be used to train the axial transformation model. The sample axial MSCT-CBCT image pairs may include a sample axial MSCT slice and a corresponding sample axial CBCT slice. At least two sample coronal MSCT-CBCT image pairs may be used to train the coronal transformation model. The sample coronal MSCT-CBCT image pairs may include a sample coronal MSCT slice and a corresponding sample coronal CBCT slice. At least two sample sagittal MSCT-CBCT image pairs may be used to train the sagittal transformation model. The sample sagittal MSCT-CBCT image pairs may include a sample sagittal MSCT slice and a sample sagittal CBCT slice.

[0121] Figure 6It is a flowchart of an exemplary process 600 for generating a conversion model of a direction according to some embodiments of the present application. In some embodiments, process 600 can be implemented as a set of instructions (e.g., an application) stored in storage device 130, memory 220, or memory 390. Processing device 120, processor 210, and / or CPU 340 can execute the set of instructions, and when executing the instructions, processing device 120, processor 210, and / or CPU 340 can be configured to execute process 600. The operations of the process shown below are for illustrative purposes only. In some embodiments, process 600 can be implemented by one or more additional operations not described and / or one or more of the operations not discussed. In addition, Figure 6 The order in which the operations of process 600 are illustrated and described below is not intended to be limiting.

[0122] In some embodiments, the conversion model of the direction can be trained offline. For example, the conversion model can be trained and stored in a storage device (e.g., storage device 130, memory 220, memory 390, etc.) of the imaging system 100. The processing device 120A can access the storage device to retrieve the conversion model for processing the 3D image. Alternatively, the conversion model can be trained in real time. For example, the processing device 120B can train the conversion model before using the conversion model.

[0123] In step 602, the processing device 120B (e.g., the training sample acquisition module 460) may acquire a plurality of training samples (or referred to as sample image pairs). The sample image pairs may include a sample target image and a corresponding sample input image for model training (e.g., a sample MSCT image as a sample target image and a sample CBCT image as a sample input image). The sample image pairs may include two corresponding images of different image qualities and / or different modalities. As used in the present disclosure, two images are considered to correspond to each other if each image includes a description of the same target or the same portion thereof. By way of example only, a first 3D image of the sample image pair may have a lower image quality and / or a first modality; a second 3D image of the sample image pair may have a higher image quality and / or a second modality. The first 3D image may include at least two first 2D images for input for training a conversion model. The second 3D image may include at least two second 2D images for expected output when training a conversion model.

[0124] By way of example only, a plurality of training samples may include a plurality of sample MSCT images and a plurality of corresponding sample CBCT images. In some embodiments, at least one sample image may be first acquired by scanning using an imaging device (e.g., an MSCT device, a CBCT device). The plurality of sample images may be acquired using the same imaging device or different imaging devices. For example, the plurality of sample MSCT images may be acquired using different MSCT devices. In some embodiments, at least one sample image may be synthesized based on an image acquired from a previous scan. In some embodiments, one or more sample input images may be synthesized based on a sample target image. By way of example only, based on a sample MSCT image acquired through a previous scan, one or more sample CBCT images may be synthesized according to a simulation operation. A sample CBCT image may be acquired by applying a simulation operation on the sample MSCT image, and the synthesized sample CBCT image and sample MSCT image may form a training sample. Exemplary procedures for simulation operations (e.g., Figure 7 In some embodiments, a plurality of training samples may be predetermined and stored in a storage device (eg, storage device 130, memory 220, memory 390, etc.) of the imaging system 100. The processing device 120B may access the storage device to obtain the plurality of training samples.

[0125] In step 604 , the processing device 120B (eg, the training module 480 ) may train a preliminary model based on a plurality of training samples to generate a conversion model.

[0126] In some embodiments, the preliminary model may be a process or an algorithm. In some embodiments, the preliminary model may be a deep learning model. For example, the preliminary model may include a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generated adversarial network (GaN) model, a CycleanGan model, a PIX2PIX model, etc., or any combination. In some embodiments, the preliminary model may be used to process 2D or 3D images. For example, the preliminary model may be a process or algorithm for processing 3D images. A plurality of training samples may be directly used to train the preliminary model, thereby obtaining a conversion model for processing 3D images. As another example, the preliminary model may be a process or algorithm for processing 2D images. Processing a plurality of training samples may obtain sample slices of a plurality of training samples. Sample slices of a plurality of training samples may be used to train the preliminary model, thereby determining a conversion model for processing 2D slices of 3D images. For each of a plurality of training samples, the training module 480 may obtain a plurality of sample slices of the direction from the sample input image of the training sample.

[0127] In some embodiments, the slice acquisition module 470 may identify a plurality of sample slices from a sample image (e.g., a sample target image, a sample input image) along a direction. For example, the slice acquisition module 470 may identify at least two planes along the direction of the sample image and a transformation matrix of each of the at least two planes. Based on an array of the sample image and a transformation matrix of each of the at least two planes, one of the at least two sample slices may be acquired. Each of at least a portion of the sample slices of the sample input image may correspond to a sample slice of the sample target image.

[0128] As an example only, the sample input image is a CBCT image, and the sample target image is an MSCT image. The sample CBCT image and the sample MSCT image may be processed in the same manner (e.g., spaced according to the same thickness along a direction) to obtain at least two sample CBCT slices and at least two sample MSCT slices. Each of at least a portion of the at least two sample CBCT slices may correspond one-to-one to one of the at least two sample MSCT slices.

[0129] In some embodiments, the training module 480 may train a preliminary model with a plurality of sample slices of the direction of the sample input image and a corresponding sample slice of the direction of the sample target image of each of the plurality of training samples. In some embodiments, the conversion model of the first direction may be trained with sample slices of one or more second directions different from the first direction. For example, the conversion model of the direction of the coronal plane of the target may be trained with sample slices in the sagittal plane of the sample target of the plurality of training samples. The training samples may include a 3D image pair, wherein one 3D image includes a plurality of first 2D images in the sagittal plane of the sample target as input to the training conversion model, and the other 3D image includes a plurality of first 2D images in the sagittal plane of the sample target as expected output of the training conversion model.

[0130] In some embodiments, the preliminary model may include at least two layers, each layer including at least two parameters. In some embodiments, at least two layers may be determined in parallel during the training process of the preliminary model. For example, the training module 480 may determine multiple layers based on multiple training samples by iterating and jointly updating the parameters of each layer of the preliminary model. Alternatively, at least two layers may be determined sequentially during the training of the preliminary model, wherein the training of a particular layer is performed only after all layers upstream are completed. For example, layer Mi (i is greater than 1) of the preliminary model may be determined based on training samples determined before determining layer Mi and determining layers M1 to Mi-1.

[0131] In some embodiments, if the termination condition is met, the training module 480 can terminate the training process to obtain the conversion model. For example, if the value of the loss function is the smallest or less than a threshold value (e.g., a constant), the termination condition can be considered to be met. Determining the loss function can measure the difference between the output slice obtained by inputting the sample input slice into the preliminary model (during the training process) and the sample target slice corresponding to the sample input slice. As another example, if the value of the loss function converges, the termination condition can be considered to be met. In some embodiments, for example, if the change in the value of the loss function in two or more consecutive iterations is equal to or less than a threshold value (e.g., a constant), it can be considered that convergence has occurred. As another example, if a certain number of iterations have been performed, the termination condition can be considered to be met. As another example, if the number of training samples used for training reaches a certain value, the termination condition can be considered to be met.

[0132] In some embodiments, when the number (or count) of new training samples exceeds a count threshold, an update of the conversion model may be triggered so that the conversion model may be optimized with the new training samples. Alternatively, the conversion model may be updated in response to receiving user input to optimize the conversion model.

[0133] In some embodiments, after determining the conversion model, the conversion model may be tested to determine whether the conversion model satisfies a predetermined condition. For example, each of the one or more test samples may include a test input image and a test target image, and the test target image may be used to test or verify the conversion model. For example, for a conversion model trained with a CBCT image and a corresponding MSCT image, at least two test samples may also include an image pair of a CBCT image and a corresponding MSCT image.

[0134] In some embodiments, at least one test image of at least two test samples may be acquired by first scanning using an imaging device. In some embodiments, at least one test image of at least two test samples may be synthesized. As described in conjunction with step 602, the test samples may be acquired similarly to the training samples, and a description thereof is not repeated here. In some embodiments, some training samples may be designated as test samples for testing the determined conversion model.

[0135] In some embodiments, for a test sample including a test input image and a test target image, at least two slices of the direction of the test input image can be input into the conversion model to obtain at least two output slices. The difference between each of the plurality of output slices and the corresponding slice of the test target image of the test sample can be determined and compared with a threshold. If the difference exceeds the threshold, it can be considered that the conversion model does not meet the predetermined condition, and further training of the conversion model may be required. If the difference is below the threshold, it can be considered that the conversion model meets the predetermined condition, and the conversion model can be used to process the 3D image.

[0136] It should be noted that the above description is only for illustration and is not intended to limit the scope of the present disclosure. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications do not depart from the scope of the present application. For example, one or more other optional operations (e.g., storage operations) can be added to the process 600. The processing device 120 can store information and / or data (e.g., sample input images and / or slices thereof, sample target images and / or slices thereof, test input images and / or slices thereof, test target images and / or slices thereof, etc.) of the storage devices (e.g., storage devices 130, memory 220, memory 390, etc.) of the imaging system 100.

[0137] Figure 7 It is an exemplary flow chart of simulation operations shown in some embodiments of the present application. In some embodiments, process 700 can be implemented as a set of instructions (e.g., an application) stored in storage device 130, memory 220, or memory 390. Processing device 120, processor 210, and / or CPU 340 can execute the set of instructions, and when executing the instructions, processing device 120, processor 210, and / or CPU 340 can be configured to execute process 700. The operations of the process shown below are for illustrative purposes only. In some embodiments, process 700 can be implemented by one or more additional operations not described and / or one or more of the operations not discussed. In addition, Figure 7 The order in which the operations of process 700 are shown in and described below is not intended to be limiting. In some embodiments, for a training sample of a plurality of training samples, a sample input image may be determined by applying process 700 on a corresponding sample target image.

[0138] In step 702, the processing device 120B (e.g., the sample synthesis module 490) may convert the sample target image into at least one 3D array. In some embodiments, the 3D array may represent the physical composition in the sample target image. For example, the 3D array may reflect the tissue type (e.g., bone, soft tissue, air, etc.), the density of the tissue in the sample target image, etc. In some embodiments, the sample target image may be converted into a 3D array representing tissue distribution (e.g., the distribution of bone tissue, soft tissue, air, etc.), a 3D array of density distribution, etc., or any combination thereof.

[0139] In step 704, the sample synthesis module 490 may synthesize a sample input image based on a sample target image. For example, the sample target image may be a sample MSCT image. The projection values ​​of the sample MSCT image may be obtained by performing a projection algorithm and a noise simulation algorithm on the sample MSCT image based at least in part on at least one 3D array.

[0140] In some embodiments, a forward projection polychromatic ray tracing algorithm may be performed on at least one 3D array at multiple projection angles using the CBCT spectrum / phase space as a beam to obtain at least one projection 2D array. For example, the MV level spectrum / phase space may be performed as a beam on at least one 3D array of sample MSCT images to obtain a MV level sample CBCT image. As another example, the KV level spectrum / phase space may be performed as a beam on at least one 3D array of sample MSCT images to obtain a KV level sample CBCT image. In some embodiments, after performing the forward projection polychromatic ray tracing algorithm, an electronic noise simulation algorithm or a quantum noise simulation algorithm may be performed on at least one projection 2D array to obtain projection values ​​of the sample MSCT image. For example, Poisson / Gaussian noise (using a random number generator) may be added to at least one projection 2D array to obtain projection values. In some embodiments, a detector energy response model and / or a detector optical model may be applied to the projection values ​​to simulate the energy and / or light signals detected by the detector of the CBCT device.

[0141] The sample synthesis module 490 can determine the synthesized sample CBCT image by reconstructing the projection values ​​using a reconstruction algorithm. In some embodiments, the projection values ​​can be reconstructed using a similar reconstruction algorithm used in a CBCT device. For example, the reconstruction algorithm can include an FDK algorithm, a variable FDK algorithm (V-FDK algorithm), an FDK-FAST algorithm, a curve-filtered FDK (C-FDK) algorithm, etc., or any combination thereof.

[0142] It should be noted that the above description is for illustration only and is not intended to limit the scope of the present disclosure. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications do not depart from the scope of the present application. For example, one or more other optional operations (e.g., storage operations) can be added elsewhere in the process 700. The processing device 120 can store information and / or data (e.g., sample target images) in a storage device (e.g., storage device 130, memory 220, memory 390, etc.) of the imaging system 100.

[0143] Example

[0144] The following examples are offered for illustrative purposes only and are not intended to be limiting.

[0145] The preliminary model may be a PIX2PIX model. A first conversion model in a first direction may be obtained by training the PIX2PIX model using training samples. The training samples may include a sample CBCT image as a sample input image and a sample MSCT image as a sample target image. Each sample CBCT image and sample MSCT image of the training samples used to train the first conversion model includes a slice in a first direction parallel to the axial plane.

[0146] The second conversion model for the second direction can be obtained by training the PIX2PIX model using training samples. The training samples include a sample CBCT image as a sample input image and a sample MSCT image as a sample target image. Each sample CBCT image and sample MSCT image of the training samples used to train the second conversion model includes a slice in the second direction parallel to the sagittal plane.

[0147] A third conversion model of the third direction can be obtained by training the PIX2PIX model using training samples. The training samples include a sample CBCT image as a sample input image and a sample MSCT image as a sample target image. Each sample CBCT image and sample MSCT image of the training samples used to train the third conversion model includes a slice of the third direction parallel to the coronal plane.

[0148] The MSCT device scans the abdomen of the sample to obtain a test MSCT image. The test MSCT image is cut along the first, second and third directions to obtain at least two test slices, which are used to test the first conversion model in the first direction, the second conversion model in the second direction and the third conversion model in the third direction. The first conversion model in the first direction, the second conversion model in the second direction and the third conversion model in the third direction are used for the image processing result as shown in FIG. Figure 14-Figure 19 shown.

[0149] Figure 8Schematic diagram of a test axial MSCT slice according to some embodiments of the present application. A test axial MSCT slice is acquired from a test MSCT image along a first direction parallel to the axial plane of the test MSCT image. Fig. 9 Schematic diagram of a test sagittal MSCT slice according to some embodiments of the present application. The test sagittal MSCT slice is acquired from the test MSCT image along a second direction parallel to the sagittal plane of the test MSCT image. Fig.10 Schematic diagram of a test coronal MSCT slice according to some embodiments of the present application. The test coronal MSCT slice is acquired from the test MSCT image along a third direction parallel to the coronal plane of the test MSCT image.

[0150] In some embodiments, the simulation operation described in process 700 is applied to the test MSCT image to synthesize the test CBCT image. At least two test CBCT slices are obtained from the test CBCT image along the first direction, the second direction, and the third direction, respectively. Fig.11 is a schematic diagram of a test axial CBCT slice according to some embodiments of the present application. Fig.12 is a schematic diagram of a test sagittal CBCT slice according to some embodiments of the present application. Fig.13 is a schematic diagram of a test coronal CBCT slice according to some embodiments of the present application.

[0151] Example 1

[0152] Applying the first transformation model to at least two test axial CBCT slices can obtain at least two processed test axial CBCT slices. Specifically, the at least two test axial CBCT slices are input into the first transformation model, and the output of the first transformation model includes the at least two processed test axial CBCT slices. An axially transformed 3D CBCT image is reconstructed based on the at least two processed test axial CBCT slices. The axial slices, sagittal slices, and coronal slices of the axially transformed 3D CBCT image are respectively as follows: Figure 14-16 shown.

[0153] like Figure 14-16 As shown, the axial slices of the axially transformed 3D CBCT image have no slice discontinuity artifacts (eg, step artifacts), while the sagittal slices and coronal slices of the axially transformed 3D CBCT image have large artifacts.

[0154] Example 2

[0155] like Fig.11 As shown, the first transformation model is applied to the test axial CBCT slice. Fig.12 As shown, the second transformation model is applied to the test sagittal CBCT slice. Fig.13As shown in FIG. 1 , the third transformation model is applied to the test coronal CBCT slice. The output of the first transformation model (processed axial CBCT slice) is shown in FIG. Fig.17 The output of the second transformation model (processed sagittal CBCT slice) is shown in Fig.18 The output of the third transformation model (processed coronal CBCT slices) is shown in Fig.19 shown.

[0156] like Figure 17-Figure 19 As shown, the processed axial CBCT slices, the processed sagittal CBCT slices, and the processed coronal CBCT slices have no slice discontinuity artifacts, and the processed (axial, sagittal, and coronal) CBCT slices are similar to the corresponding test (axial, sagittal, and coronal) MSCT slices in terms of contrast, contrast-to-noise ratio, or resolution.

[0157] The basic concepts have been described above. Obviously, for those of ordinary skill in the art who have read this application, the above invention disclosure is only for example and does not constitute a limitation of this application. Although not explicitly stated here, those of ordinary skill in the art may make various modifications, improvements and amendments to this application. Such modifications, improvements and amendments are suggested in this application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of this application.

[0158] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, the terms "one embodiment", "an embodiment" and / or "some embodiments" mean that the specific features, structures or characteristics described in conjunction with the embodiments are included in at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that "an embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0159] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product, or combination of substances, or any new and useful improvement thereof. Therefore, various aspects of the present disclosure may be implemented entirely by hardware, software (including firmware, resident software, microcode, etc.), or a combination of software and hardware, which are generally referred to as "units," "modules," or "systems" in the present disclosure. In addition, various aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0160] A non-transitory computer readable signal medium may include a propagated data signal containing a computer readable program code, for example, in baseband or as part of a carrier wave. Such propagated signals may be in a variety of forms, including electromagnetic, optical, etc., or any suitable combination. A computer readable signal medium may be any computer readable medium other than a computer readable storage medium, which may be connected to an instruction execution system, device, or apparatus to communicate, propagate, or transmit a program for use. The program code on a computer readable signal medium may be propagated via any suitable medium, including radio, cable, fiber optic cable, RF, etc., or any combination of the above.

[0161] The computer program code for performing the operation of the present disclosure can be written in any combination of one or more programming languages, including target-oriented programming languages, such as java, Scala, Simal Talk, Effele, JADE, Emerald, C++, C#, VB.NET, Python, etc., or any combination thereof, traditional process programming languages, such as "C" programming language, Visual Basic, Fortran, Perl, COBOL, PHP, ABAP, dynamic programming languages, such as Python, Ruby and Groovy, or other programming languages. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on the remote computer, or run completely on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0162] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0163] Similarly, it should be understood that in the foregoing description of the embodiments of the present application, various features are sometimes combined together in a single embodiment, figure, or description thereof to simplify the present application and facilitate understanding of one or more of the various invention embodiments. However, this approach of the present application should not be interpreted as reflecting the intention that the claimed object material to be scanned requires more features than those expressly stated in each claim. On the contrary, the subject matter of the invention should have fewer features than the single embodiment described above.

[0164] In some embodiments, the numbers used to describe and request the quantity, attributes, etc. of an application should be understood to be modified in some cases by the term "approximately" or "substantially" in some cases. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the desired characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of the present application are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0165] Each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, documents, articles, and / or the like, cited herein is hereby incorporated by reference in its entirety for all purposes, except any prosecution document history that is related to this document, is inconsistent with or conflicts with this document, or would have the effect of limiting the broadest scope of the claims now or later related to this document. For example, if there is any inconsistency or conflict between the description, definition, and / or use of terminology associated with any incorporated material and the terminology associated with this document, the description, definition, and / or terminology used in this document shall control.

[0166] Finally, it should be understood that the embodiments described in this application are only used to illustrate the principles of the embodiments of the present application. Other variations may also fall within the scope of the present application. Therefore, as an example and not a limitation, the alternative configurations of the embodiments of the present application may be considered to be consistent with the teachings of the present application. Accordingly, the embodiments of the present application are not limited to the embodiments explicitly introduced and described in the present application.

Claims

1. A system for processing 3D images, characterized in that: include: at least one storage device comprising a set of instructions for processing a 3D image; as well as at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to perform operations comprising: Along one direction of the 3D image, acquiring at least two slices in the direction from the 3D image; Processing the at least two slices in the direction using a transformation model, acquiring the at least two processed slices in the direction, wherein the at least two processed slices in the direction are used to determine a transformed 3D image in the direction, wherein the transformed 3D image in the direction has improved image quality and / or a different modality relative to the 3D image, wherein acquiring the transformation model comprises: Acquire at least two training samples, each of the training samples comprising sample images of different image qualities and / or different modalities; and Based on the at least two training samples, a preliminary model is trained to obtain the conversion model.

2. The system according to claim 1, characterized in that The at least one processor is configured to cause the system to further perform the operations further comprising: Based on the at least two processed slices, at least one second slice of the direction to be displayed is acquired.

3. The system according to claim 1, characterized in that The at least one processor is configured to cause the system to further perform the operations further comprising: along a second direction of the 3D image, Acquire at least two slices in the second direction from the 3D image; and The at least two slices in the second direction are processed using a second transformation model to obtain at least two processed slices in the second direction, wherein the transformation model is different from the second transformation model and the direction is different from the second direction.

4. The system according to claim 3, characterized in that The at least one processor is configured to cause the system to further perform the operations including: Based on the at least two processed slices of the second direction, a transformed 3D image of the second direction is determined.

5. The system according to claim 3, characterized in that The at least one processor is configured to cause the system to further perform the operations further comprising: At least one second slice in the second direction to be displayed is acquired based on the at least two processed slices in the second direction.

6. The system according to claim 1, characterized in that Each of the training samples includes a sample multi-slice computed tomography (MSCT) image and / or a sample CBCT image.

7. The system according to claim 6, characterized in that Based on the at least two training samples, training the preliminary model comprises: For each of the at least two training samples, Acquire at least two sample CBCT slices in the direction from the sample CBCT image of the training sample; Acquire at least two sample MSCT slices in the direction from the sample MSCT image of the training sample; and The preliminary model is trained using the at least two sample CBCT slices in the orientation as input and the at least two sample MSCT slices in the orientation as expected output.

8. The system according to claim 6, characterized in that For at least one of the at least two training samples, the sample CBCT image of the training sample is determined by applying a simulation operation on the sample MSCT image of the training sample.

9. The system according to claim 8, characterized in that The simulation operation includes: converting the sample MSCT image into at least one 3D array; Obtaining projection values ​​of the sample MSCT image by performing a projection algorithm and a noise simulation algorithm on the sample MSCT image based at least in part on the at least one 3D array; and The sample CBCT image is acquired by reconstructing the projection values ​​using a reconstruction algorithm.

10. The system according to claim 1, characterized in that The direction is associated with at least one of an axial plane, a coronal plane, or a sagittal plane of the 3D image.

11. The system according to claim 1, characterized in that The transformation model is a single transformation model for processing slices in at least two directions, and the single transformation model is obtained by the following process: The preliminary model is trained using at least two sample slices in the at least two directions.

12. The system according to claim 1, characterized in that The at least one processor is configured so that the system further performs the operations further comprising: along a second direction of the 3D image, Acquire at least two slices in the second direction from the 3D image; and The at least two slices in the second direction are processed using the transformation model to obtain at least two processed slices in the second direction, the direction being different from the second direction.

13. A system, characterized in that: include: at least one storage device, the at least one storage device comprising a set of instructions; as well as at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to perform operations comprising: identifying at least two second slices corresponding to the different directions from at least two converted 3D images in different directions, the at least two converted 3D images being obtained by processing the first slices of the 3D images in the different directions; and The at least two second slices corresponding to the different directions are sent to at least one terminal for display, wherein the at least two second slices are determined in the following manner: Acquire at least two first slices in a direction of the 3D image from the 3D image; Processing the at least two first slices in the direction using a transformation model to obtain the at least two second slices in the direction, wherein the at least two second slices in the direction are used to determine a transformed 3D image in the direction, wherein the transformed 3D image in the direction has improved image quality and / or a different modality relative to the 3D image, wherein obtaining the transformation model comprises: Acquire at least two training samples, each of the training samples comprising sample images of different image qualities and / or different modalities; and Based on the at least two training samples, a preliminary model is trained to obtain the conversion model.

14. The system according to claim 13, characterized in that The operations also include: A notification is generated and sent to the at least one terminal, wherein the at least two second slices are obtained from the at least two converted 3D images, wherein each of the converted 3D images is obtained by processing a slice in one of the different directions.

15. The system according to claim 13, characterized in that The at least two transformed 3D images are obtained by: Processing the slices of the different directions using at least two transformation models, each of the at least two transformation models corresponding to one direction of the different directions; and Each of the at least two transformed 3D images is determined based on the at least two processed slices of the corresponding direction.

16. The system according to claim 15, characterized in that The at least two conversion models corresponding to the at least two directions are different from each other.

17. The system according to claim 13, characterized in that The at least two transformed 3D images are obtained by: Processing the slices in different directions using a transformation model; and Each of the at least two transformed 3D images is determined based on the at least two processed slices of the corresponding direction.

18. The system according to claim 13, characterized in that The visualization of multiple view planes of the 3D image along different directions is generated by synchronously displaying a second slice of the transformed 3D image along the corresponding view planes, wherein the multiple view planes of the multiple directions intersect at a selected point in the 3D image.

19. The system according to claim 18, characterized in that The selected point is interactively selected by a user, or automatically determined based on a predetermined trajectory.

20. An image processing system, characterized in that: include: at least one storage device, the at least one storage device comprising a set of instructions; as well as at least one processor in communication with at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to perform operations comprising: Acquire megavoltage CBCT images; Along one direction of the megavoltage CBCT image, acquiring at least two slices in the direction from the megavoltage CBCT image; Processing the at least two slices in the direction using the conversion model, acquiring the at least two processed slices in the direction, wherein the at least two processed slices in the direction are used to determine a kilovoltage MSCT image in the direction, wherein the kilovoltage MSCT image in the direction has improved image quality and / or a different modality relative to the megavoltage CBCT image, wherein acquiring the conversion model comprises: Acquire at least two training samples, each of the training samples comprising a sample megavoltage CBCT image and a sample kilovoltage MSCT image; and Based on the at least two training samples, a preliminary model is trained to obtain the conversion model.

21. The system according to claim 20, characterized in that The operations also include: determining a region of interest (ROI) of the kV MSCT image; A treatment plan is acquired, wherein the treatment plan is determined based on the ROI of the kilovoltage MSCT image.

22. The system according to claim 21, characterized in that The operations also include: Determining a registration result by registering the kV MSCT image with the planning image; and A determination is made as to whether to modify the treatment plan based on the registration result.

Images