Image registration system and method

Abstract

Systems and methods for image registration are provided. The method may include acquiring a reference image of an object including a target volume (510); determining a registration mask associated with a region of interest (ROI) based on the reference image (520), the ROI including at least a portion of the target volume; acquiring a target image of the object including the target volume (530); and performing image registration of the reference image and the target image based on the registration mask (540).

Description

Technical Field

[0001] This application generally relates to image processing, and more specifically, to systems and methods for image registration. Background Technology

[0002] Image-guided radiotherapy (IGRT) is a widely used radiotherapy modality for the clinical treatment of various cancers. During radiotherapy, it is often necessary to register treatment images acquired before or during radiotherapy with pre-generated planning images to plan the radiotherapy and ensure accurate delivery of the treatment to the target volume (e.g., the tumor of the subject, e.g., the patient), according to the treatment plan. However, some interfering features (e.g., moving organs or tissues) can significantly affect the registration results between treatment and planning images, thereby impacting subsequent treatment and / or clinical diagnosis. Therefore, there is a need to develop systems and methods for image registration to remove or reduce the influence of interfering structures, thereby improving registration accuracy. Summary of the Invention

[0003] According to one aspect of this application, a system for image registration is provided. The system may include at least one storage device storing executable instructions and at least one processor communicating with said at least one storage device. When the executable instructions are executed, the at least one processor is configured to cause the system to perform operations. The operations may include acquiring a reference image of a scanned object, the reference image including a target volume. The operations may further include determining, based on the reference image, a registration mask associated with a region of interest (ROI), the ROI including at least a portion of the target volume. The operations may further include acquiring a target image of the scanned object, the target image including the target volume. The operations may further include performing image registration on the reference image and the target image based on the registration mask.

[0004] In some embodiments, in order to determine a registration mask associated with a Region of Interest (ROI) based on a reference image, the operation may include generating an intermediate image by removing image information other than the ROI from the reference image; and determining the registration mask associated with the ROI based on the intermediate image.

[0005] In some embodiments, in order to determine a registration mask associated with a ROI based on an intermediate image, the operation may include filtering the intermediate image; and determining the registration mask associated with the ROI based on the filtered intermediate image.

[0006] In some embodiments, in order to determine a registration mask associated with a Region of Interest (ROI) based on a reference image, the operation may include determining a preliminary mask including the ROI based on the reference image; identifying one or more interfering structures in the ROI that are at least partially located in the reference image; determining an auxiliary mask including the one or more interfering structures; and determining the registration mask associated with the ROI based on the preliminary mask and the auxiliary mask.

[0007] In some embodiments, in order to identify one or more interfering structures in a ROI that is at least partially located in a reference image, the operation may include automatically segmenting the one or more interfering structures from the reference image according to preset rules.

[0008] In some embodiments, the preset rules include at least one of the structure type, structure label, or time information associated with the reference image.

[0009] In some embodiments, one or more interfering structures include at least one of musculoskeletal organs or musculoskeletal tissues.

[0010] In some embodiments, the musculoskeletal organ or musculoskeletal tissue is determined based on reference film images.

[0011] In some embodiments, the musculoskeletal organ or musculoskeletal tissue is determined based on a target film image.

[0012] In some embodiments, the kinetic organ or the kinetic tissue is determined based on the motion characteristics of the organ or tissue identified in the target image or the reference image.

[0013] In some embodiments, one or more interfering structures include at least one of the heart, lungs, diaphragm, bladder, or rectum.

[0014] In some embodiments, in order to perform image registration of a reference image and a target image based on a registration mask, the operation may include determining a mask reference image by applying the registration mask to the reference image; determining a mask target image by applying the registration mask to the target image; and determining one or more offset parameters based on the mask reference image and the mask target image.

[0015] In some embodiments, in order to determine one or more offset parameters based on a mask reference image and a mask target image, for each of the one or more target regions in the mask target image, the operation may include determining a similarity value between the target region and a corresponding reference region in the mask reference image for each of the one or more target regions in the mask target image; and determining the one or more offset parameters based on one or more similarity values ​​corresponding to the one or more target regions.

[0016] In some embodiments, the reference image and the target image are acquired at the same respiratory phase.

[0017] In some embodiments, the reference image is a planning image associated with treatment; and the target image is a treatment image before or during treatment.

[0018] In some embodiments, the planning image is a digital reconstructed radiograph (DRR); and the treatment image is an X-ray image.

[0019] In some embodiments, each of the reference image and the target image is a 3D image or a 4D image.

[0020] According to another aspect of this application, an image registration method is provided. This method can be implemented on a computing device having at least one processor and at least one storage device. The method may include acquiring a reference image of a scanned object, the reference image including a target volume; determining a registration mask associated with a region of interest (ROI) based on the reference image, the ROI including at least a portion of the target volume; acquiring a target image of the scanned object, the target image including the target volume; and performing image registration on the reference image and the target image based on the registration mask.

[0021] According to another aspect of this application, a non-transitory computer-readable medium is provided, comprising at least one set of instructions, wherein, when executed by at least one processor of a computing device, the at least one set of instructions instructs at least one processor to perform a method. The method may include acquiring a reference image of a scanned object, the reference image including a target volume; determining, based on the reference image, a registration mask associated with a region of interest (ROI), the ROI including at least a portion of the target volume; acquiring a target image of the scanned object, the target image including the target volume; and performing image registration on the reference image and the target image based on the registration mask.

[0022] Additional features will be set forth in part in the following description, and in part will become apparent to those skilled in the art upon examination of the following and accompanying drawings, or may be learned by the generation or operation of embodiments. The features of this application may be implemented and obtained by practice or use of various aspects of the methods, means, and combinations set forth in the detailed examples discussed below. Attached Figure Description

[0023] This application will be further described through exemplary embodiments. These exemplary embodiments will be described in detail with reference to the accompanying drawings. The drawings are not drawn to scale. These embodiments are non-limiting exemplary embodiments, in which the same numbers in the figures denote similar structures, wherein:

[0024] Figure 1 This is a schematic diagram illustrating an exemplary medical system according to some embodiments of this application;

[0025] Figure 2 This is a schematic diagram illustrating exemplary hardware and / or software components of an exemplary computing device according to some embodiments of this application;

[0026] Figure 3 This is a schematic diagram illustrating exemplary hardware and / or software components of an exemplary mobile device according to some embodiments of this application;

[0027] Figure 4 This is a block diagram illustrating an exemplary processing apparatus according to some embodiments of this application;

[0028] Figure 5 This is a flowchart illustrating an exemplary process for image registration according to some embodiments of this application;

[0029] Figure 6 This is a flowchart illustrating an exemplary process for determining a registration mask according to some embodiments of this application;

[0030] Figure 7 This is a flowchart illustrating an exemplary process for registering a reference image with a target image based on a registration mask, according to some embodiments of this application;

[0031] Figure 8A An exemplary image registration process based on a registration mask between a planning image and a treatment image during treatment of a chest region is shown according to some embodiments of this application;

[0032] Figure 8B An exemplary image registration process based on a registration mask between a planning image and a treatment image during treatment of a chest region is shown according to some embodiments of this application. Detailed Implementation

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. However, those skilled in the art should understand that this application can be implemented without these details. In other instances, to avoid unnecessarily obscuring various aspects of this application, well-known methods, processes, systems, components, and / or circuits have been described at a higher level. It will be apparent to those skilled in the art that various changes can be made to the disclosed embodiments, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the embodiments shown, but conforms to the broadest scope consistent with the scope of the claims.

[0034] The terminology used in this application is for the purpose of describing particular exemplary embodiments only and is not restrictive. The singular forms “a,” “an,” and “the” used in this application may also include the plural forms unless the context clearly indicates otherwise. It should be understood that the terms “comprising” and “including” as used in this specification only indicate the presence of the stated features, integers, steps, operations, components, and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts, and / or combinations thereof.

[0035] It should be understood that the terms “system,” “engine,” “unit,” “module,” and / or “block” used herein are a method of distinguishing different components, elements, parts, sections, or components at different levels in ascending order. However, if these terms serve the same purpose, they may be replaced by another expression.

[0036] Generally, the terms "module," "unit," or "block" as used herein refer to logic embodied in hardware or firmware, or a collection of software instructions. The modules, units, or blocks described herein can be implemented as software and / or hardware and can be stored on any type of non-transitory computer-readable medium or other storage device. In some embodiments, software modules / units / blocks can be compiled and linked into an executable program. It should be understood that software modules can be called from other modules / units / blocks or from themselves, and / or can be called in response to detected events or interrupts. Configured for use in computing devices (e.g., Figure 2Software modules / units / blocks executing on the processor 210 shown herein may be provided on a computer-readable medium or as digital downloads (and may be initially stored in a compressed or installable format, requiring installation, decompression, or decryption before execution). The software code herein may be stored, in part or in whole, in the storage device of the computing device performing the operation and applied in the operation of the computing device. Software instructions may be embedded in firmware, such as EPROM. It should also be understood that hardware modules / units / blocks may be included in connected logical components, such as gates and flip-flops, and / or may include programmable units, such as programmable gate arrays or processors. The modules / units / blocks or computing device functions described herein may be implemented as software modules / units or blocks, but may be represented in hardware or firmware. The modules / units / blocks described herein refer to logical modules / units / blocks that may be combined with other modules / units / blocks or divided into sub-modules / sub-units / sub-blocks, although they are physical organizations or storage devices. This description may apply to a system, an engine, or a part thereof.

[0037] It should be understood that when a unit, engine, module, or block is referred to as being "placed," "connected to," or "coupled to" another unit, engine, module, or block, it may be directly on, connected to, or coupled to another unit, engine, component, or block, unless the context clearly indicates otherwise. In this application, the term "and / or" may include any one or more of the relevant listed items or a combination thereof.

[0038] These and other features, characteristics, functions and operating methods of related structural elements, as well as component assembly and manufacturing economics, will become more apparent from the following description of the accompanying drawings, which form part of this application specification. However, it should be understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this application. It should be understood that the drawings are not drawn to scale.

[0039] The term "image" in this application is used to refer to image data (e.g., scanned data, projected data) and / or images of various forms, including two-dimensional (2D) images, three-dimensional (3D) images, four-dimensional (4D) images, etc. The terms "pixel" and "voxel" in this application are used interchangeably to refer to elements of an image. The terms "region," "location," and "area" in this application can refer to the location of an anatomical structure shown in an image or the actual location of an anatomical structure existing within or on the surface of a target object, because an image can indicate the actual location of a specific anatomical structure existing within or on the surface of a target object.

[0040] This document provides non-invasive biomedical imaging / treatment systems and methods for purposes such as disease diagnosis, disease treatment, or research. In some embodiments, the system may include an imaging system. The imaging system may include a single-modal system and / or a multimodal system. As used herein, the term "modality" broadly refers to the imaging or treatment methods or techniques for collecting, generating, processing, and / or analyzing imaging information of a subject or treating a subject. A single-modal system may include, for example, an ultrasound imaging system, an X-ray imaging system, a computed tomography (CT) system, a magnetic resonance imaging (MRI) system, an ultrasound contrast imaging system, a positron emission tomography (PET) system, an optical coherence tomography (OCT) imaging system, an ultrasound (US) imaging system, an intravascular ultrasound (IVUS) imaging system, a near-infrared spectroscopy (NIRS) imaging system, and any combination thereof. Multimodal systems may include, for example, X-ray imaging-magnetic resonance imaging (X-ray MRI) systems, positron emission tomography-X-ray imaging (PET-X-ray) systems, single-photon emission computed tomography-magnetic resonance imaging (SPECT-MRI) systems, positron emission tomography-computed tomography (PET-CT) systems, C-arm systems, positron emission tomography-magnetic resonance imaging (PET-MR) systems, digital subtraction angiography-magnetic resonance imaging (DSA-MRI) systems, etc. In some embodiments, the medical system may include a treatment system. The treatment system may include a treatment planning system (TPS), image-guided radiotherapy (IGRT), etc. Image-guided radiotherapy (IGRT) may include treatment equipment and imaging equipment. The treatment equipment may include linear accelerators, cyclotron accelerators, synchrotrons, etc., configured to perform radiotherapy on the scanned object. The treatment equipment may include accelerators of various particle types, including, for example, photons, electrons, protons, or heavy ions. Imaging equipment may include MRI scanners, CT scanners (e.g., cone-beam computed tomography (CBCT) scanners), digital radiography (DR) scanners, electronic field imaging devices (EPID), etc. It should be noted that the medical systems described below are for illustrative purposes only and are not intended to limit the scope of this application.

[0041] In this application, the scanned object may include biological objects and / or non-biological objects. Biological objects may be humans, animals, plants, or specific parts, organs, and / or tissues thereof. For example, the scanned object may include the head, neck, chest, heart, stomach, blood vessels, soft tissue, tumors, nodules, etc., or any combination thereof. In some embodiments, the object may be an artificial composition of organic and / or inorganic substances, with or without life. The terms "object" and "scanned object" are used interchangeably in this application.

[0042] In this application, for the sake of brevity, a representation of an object (e.g., a patient, a scanned object, or a portion thereof) in an image may be referred to as an "object." For example, for the sake of brevity, a representation of an organ or tissue (e.g., heart, liver, lung) in an image may be referred to as an organ or tissue. Furthermore, for the sake of brevity, an image including a representation of an object may be referred to as an image of the object or an image including the object. Further still, for the sake of brevity, an operation performed on a representation of an object in an image may be referred to as an operation performed on the object. For example, for the sake of brevity, segmentation of an image including a representation of an organ or tissue from an image may be referred to as segmentation of the organ or tissue.

[0043] One aspect of this application relates to systems and methods for image registration. For example, the system can acquire a reference image including an object with a target volume. The system can also determine a registration mask associated with a region of interest (ROI) based on the reference image. The ROI may include at least a portion of the target volume. The system can also acquire a target image including the object with the target volume. The system can also perform image registration between the reference image and the target image based on the registration mask. Therefore, the system can register the reference image with the target image based on a clinically relevant region (i.e., an ROI including at least a portion of the target volume), which can improve the accuracy of image registration.

[0044] In some embodiments, one or more interfering structures (e.g., moving organs or tissues) can be excluded from the ROI to improve registration accuracy. For example, an auxiliary mask including one or more interfering structures can be determined and used to exclude one or more interfering structures during image registration. Accordingly, the accuracy of registration can be effectively improved, and the efficiency and / or accuracy of diagnosis and / or treatment based thereon can be effectively improved.

[0045] Figure 1 This is a schematic diagram illustrating an exemplary medical system according to some embodiments of this application. For example... Figure 1 As shown, medical system 100 may include medical device 110, network 120, terminal device 130, processing device 140, and storage device 150. In some embodiments, two or more components of medical system 100 may be interconnected and / or communicate with each other via a wireless connection (e.g., network 120), a wired connection, or a combination thereof. Components of medical system 100 may be connected in various ways. By way of example only, medical device 110 may be connected to processing device 140 via network 120 or directly. As another example, storage device 150 may be connected to processing device 140 via network 120 or directly.

[0046] Medical device 110 can be configured to acquire image data of an object and / or treat the object. For ease of explanation, a medical device that combines image acquisition and treatment functions will be used as an example. Therefore, medical device 110 may include imaging device 111, treatment device 112, and worktable 113.

[0047] Imaging device 111 can be configured to acquire images of a subject before, during, and / or after radiotherapy. In some embodiments, imaging device 111 may include a computed tomography (CT) device (e.g., cone-beam computed tomography (CBCT) device, fan-beam computed tomography (FBCT) device), ultrasound imaging device, fluorescence fluoroscopy imaging device, magnetic resonance imaging (MRI) device, single-photon emission computed tomography (SPECT) device, positron emission tomography (PET) device, X-ray imaging device, etc., or any combination thereof. For ease of illustration, this application uses a CT device as an exemplary imaging device 111.

[0048] In some embodiments, the imaging device 111 may include an imaging radiation source, a detector, and a rack. The imaging radiation source and detector may be mounted on the rack. The imaging radiation source may emit radiation toward the scanned object. The detector may detect radiation events (e.g., X-ray photons, gamma-ray photons) emitted from the imaging region of the imaging device 111. In some embodiments, the detector may include one or more detector units. The detector unit may include detectors (e.g., a gadolinium oxysulfide detector), gas detectors, etc. The detector unit may include single-row detectors and / or multi-row detectors.

[0049] The treatment device 112 can be configured to perform radiotherapy on a scanned object. In some embodiments, the treatment device 112 may include a therapeutic radiation source, a gantry, and a collimator. The therapeutic radiation source can be configured to emit therapeutic radiation toward the scanned object. In some embodiments, the therapeutic radiation source may include a linear accelerator (LINAC). The collimator can be configured to control the shape of the therapeutic radiation generated by the therapeutic radiation source.

[0050] In some embodiments, the imaging device 111 may be spaced a distance from the treatment device 112. In some embodiments, the frame of the imaging device 111 and the frame of the treatment device 112 may share a rotation axis. The scanned object may be placed at different locations on the worktable 113 for imaging and treatment. In some embodiments, the imaging radiation source and the treatment radiation source may be integrated into a single radiation source for imaging and / or treating the scanned object. In some embodiments, the imaging device 111 and the treatment device 112 may use the same frame. For example, the treatment radiation source may be mounted on the frame of the imaging device 111. The scanned object may be placed on the worktable 113 for treatment and / or imaging. In some embodiments, the worktable 113 may be aligned in a specific direction (e.g., Figure 1 The coordinate system 160 shown (Y-axis direction) moves between the treatment device 112 and the imaging device 111.

[0051] Network 120 may include any suitable network that can facilitate the exchange of information and / or data between the medical system 100 and the medical system 100. In some embodiments, one or more components of the medical system 100 (e.g., medical device 110, processing device 140, storage device 150, or terminal device 130) may transmit information and / or data with one or more other components of the medical system 100 via network 120. For example, processing device 140 may acquire image data from medical device 110 via network 120. As another example, processing device 140 may acquire user instructions from terminal device 130 via network 120. Network 120 may be or include public networks (e.g., the Internet), private networks (e.g., local area networks (LANs)), wired networks, wireless networks (e.g., 802.11 networks, Wi-Fi networks), Frame Relay networks, virtual private networks (VPNs), satellite networks, telephone networks, routers, hubs, switches, server computers, and / or any combination thereof. For example, network 120 may include wired networks, fiber optic networks, telecommunications networks, intranets, wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth. TM Network, ZigBee TM A network, a near-field communication (NFC) network, or any combination thereof. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include wired and / or wireless network access points, such as base stations and / or internet exchange points, through which one or more components of medical system 100 may connect to network 120 to exchange data and / or information.

[0052] Terminal device 130 can connect to and / or communicate with medical device 110, processing device 140, and / or storage device 150. For example, terminal device 130 can acquire a reference image including a region of interest (ROI) from processing device 140. As another example, terminal device 130 can enable user interaction with medical system 100. In some embodiments, terminal device 130 may include mobile device 131, tablet computer 132, laptop computer 133, etc., or any combination thereof. Mobile device 131 may include mobile phone, personal digital assistant (PDA), gaming device, navigation device, point-of-sale (POS) device, laptop computer, tablet computer, desktop, etc., or any combination thereof. In some embodiments, terminal device 130 may include input devices, output devices, etc. Input devices may include alphanumeric keys and other keys that can be input via a keyboard, touchscreen (e.g., with haptic or haptic feedback), voice input, eye-tracking input, brain monitoring system, or any other similar input mechanism. Input information received through the input devices may be transmitted to processing device 140 for further processing, for example, via a bus. Other types of input devices may include cursor control devices, such as a mouse, trackball, or arrow keys. Output devices may include a display, speakers, printer, or combinations thereof. In some embodiments, terminal device 130 may be part of processing device 140.

[0053] Processing device 140 can process data and / or information acquired from medical device 110, storage device 150, terminal device 130, or other components of medical system 100. For example, processing device 140 can align a reference image with a target image based on a registration mask. As another example, processing device 140 can remove one or more interfering structures from a reference image and generate an auxiliary mask. Processing device 140 can generate a registration mask based on the auxiliary mask. In some embodiments, processing device 120 may include a central processing unit (CPU), digital signal processor (DSP), system-on-a-chip (SoC), microcontroller unit (MCU), etc., or any combination thereof. In some embodiments, processing device 140 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processing device 140 may be local or remote relative to medical system 100. For example, processing device 140 may access information and / or data from medical device 110, storage device 150, and / or terminal device 130 via network 120. As another example, processing device 140 can be directly connected to medical device 110, terminal device 130, and / or storage device 150 to access information and / or data. In some embodiments, processing device 140 can be implemented on a cloud platform. For example, the cloud platform may include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, and cross-cloud, multi-cloud, etc., or combinations thereof. In some embodiments, processing device 140 may be composed of devices having, for example, combined with... Figure 2 The computing device 200 implements one or more of the described components.

[0054] Storage device 150 may store data, instructions, and / or any other information. In some embodiments, storage device 150 may store data acquired from processing device 140, terminal device 130, and / or storage device 150. In some embodiments, storage device 150 may store data and / or instructions used by processing device 140 to perform or execute the exemplary methods described in this application. In some embodiments, storage device 150 may include mass storage devices, removable storage devices, volatile read-write memory, read-only memory (ROM), etc., or any combination thereof. Exemplary mass storage devices may include disks, optical disks, solid-state drives, etc. Exemplary removable storage devices may include flash drives, floppy disks, optical disks, memory cards, compact disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM), and zero-capacitance random access memory (Z-RAM), etc. Exemplary ROMs may include mask ROMs (MROMs), programmable ROMs (PROMs), erasable programmable ROMs (EPROMs), electrically erasable programmable ROMs (EEPROMs), optical disc ROMs (CD-ROMs), and digital universal disk read-only memories, etc. In some embodiments, storage device 150 may be implemented on a cloud platform as described elsewhere in this application.

[0055] In some embodiments, storage device 150 may be connected to network 120 to communicate with one or more other components of medical system 100 (e.g., processing device 140, terminal device 130). One or more components of medical system 100 may access data or instructions stored in storage device 150 via network 120. In some embodiments, storage device 150 may be part of processing device 140.

[0056] In some embodiments, such as Figure 1 As shown, a coordinate system 160 can be provided in the medical system 100. The coordinate system 160 may include an X-axis, a Y-axis, and a Z-axis. Figure 1The X and Y axes shown can be horizontal, while the Z axis can be vertical. As shown, the X axis can be parallel to the transverse direction of the worktable 113, the Y axis can be parallel to the longitudinal direction of the worktable 113, and the Z axis can be parallel to the vertical direction of the worktable 113. The origin of coordinate system 160 can be located at any suitable location. For example, the origin can be located at the isocenter of the LINAC of the treatment device 112, and coordinate system 160 can be referred to as the RT coordinate system. As another example, the imaging device 111 can be a CT device. The origin of coordinate system 160 can be located at the rotation center of the gantry of the CT device, and coordinate system 160 can be referred to as the CT coordinate system. In some embodiments, the origin of coordinate system 160 can be determined by the operator. In some embodiments, the origin of coordinate system 160 can be determined by the default settings of the medical system 100.

[0057] The above description is intended to be illustrative and not to limit the scope of this application. Many alternatives, modifications, and variations will be apparent to those skilled in the art. Features, structures, methods, and other characteristics of the exemplary embodiments described herein can be combined in various ways to obtain additional and / or alternative exemplary embodiments. However, such variations and modifications do not depart from the scope of this application.

[0058] Figure 2 This is a schematic diagram illustrating exemplary hardware and / or software components of an exemplary computing device according to some embodiments of this application. In some embodiments, processing device 140 may be implemented on computing device 200. Figure 2 As shown, the computing device 200 may include a processor 210, a memory 220, an input / output (I / O) 230, and a communication port 240.

[0059] Processor 210 can execute computer instructions (e.g., program code) and perform the functions of processing device 140 according to the techniques described herein. Computer instructions may include, for example, routines, programs, objects, components, data structures, procedures, modules, and functions that perform the specific functions described herein. For example, processor 210 can process image data acquired from medical device 110, terminal device 130, storage device 150, and / or any other component of medical system 100. In some embodiments, processor 210 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), central processing units (CPUs), graphics processing units (GPUs), physical processors (PPUs), microcontroller units, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), advanced RISC machines (ARMs), programmable logic devices (PLDs), any circuitry or processor capable of performing one or more functions, and any combination thereof.

[0060] For illustrative purposes only, only one processor is described in computing device 200. However, it should be noted that computing device 200 in this application may also include multiple processors. Therefore, the operations and / or methods performed by one processor as described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of computing device 200 in this application performs operations A and B, it should be understood that operations A and B may also be performed jointly or individually by two or more different processors in computing device 200 (e.g., the first processor performs operation A, the second processor performs operation B, or the first and second processors jointly perform operations A and B).

[0061] Memory 220 may store data / information acquired from medical device 110, terminal device 130, storage device 150, and / or any other component of medical system 100. In some embodiments, memory 220 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), or any combination thereof. In some embodiments, memory 220 may store one or more programs and / or instructions to perform the exemplary methods described in this application. For example, memory 220 may store a program for processing device 140 to determine a registration mask based on a reference image of an object.

[0062] I / O 230 can input and / or output signals, data, information, etc. In some embodiments, I / O 230 can enable user interaction with processing device 140. In some embodiments, I / O 230 may include input devices and output devices. Exemplary input devices may include a keyboard, mouse, touchscreen, microphone, etc., or combinations thereof. Exemplary output devices may include display devices, speakers, printers, projectors, etc., or combinations thereof. Exemplary display devices may include liquid crystal displays (LCDs), light-emitting diode (LED) based displays, flat panel displays, curved screens, television equipment, cathode ray tubes (CRTs), touchscreens, etc., or combinations thereof.

[0063] Communication port 240 can be connected to a network (e.g., network 120) to facilitate data communication. Communication port 240 can establish a connection between processing device 140 and medical device 110, terminal device 130, and / or storage device 150. This connection can be a wired connection, a wireless connection, any other communication connection that enables data transmission and / or reception, and / or any combination of these connections. Wired connections may include, for example, cables, optical fibers, telephone lines, etc., or any combination thereof. Wireless connections may include, for example, Bluetooth. TM Link, Wi-Fi TM Link, WiMax TM Link, WLAN link, ZigBeeTM The communication port 240 may be a link, a mobile network link (e.g., 3G, 4G, 5G), or a combination thereof. In some embodiments, the communication port 240 may be and / or include a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port 240 may be a specially designed communication port. For example, the communication port 240 may be designed according to the Medical Digital Imaging and Communication (DICOM) protocol.

[0064] Figure 3 This is a schematic diagram illustrating exemplary hardware and / or software components of an exemplary mobile device according to some embodiments of this application. In some embodiments, one or more components of the medical system 100 (e.g., terminal device 130 and / or processing device 140) may be implemented on the mobile device 300.

[0065] like Figure 3 As shown, the mobile device 300 may include a communication platform 310, a display 320, a graphics processing unit (GPU) 330, a central processing unit (CPU) 340, I / O 350, memory 360, and storage 390. In some embodiments, any other suitable components, including but not limited to a system bus or controller (not shown), may also be included within the mobile device 300. In some embodiments, a mobile operating system 370 (e.g., iOS) TM Android TM Windows Phone TM One or more applications 380 can be loaded from storage 390 into memory 360 for execution by CPU 340. Application 380 may include a browser or any other suitable mobile application for receiving and presenting information related to image processing or other information from processing device 140. User interaction with the information flow can be achieved through I / O 350 and provided to processing device 140 and / or other components of medical system 100 via network 120.

[0066] To implement the various modules, units, and functions described in this application, a computer hardware platform may be used as the hardware platform for one or more of the components described herein. A computer with user interface elements may be used to implement a personal computer (PC) or any other type of workstation or terminal device. If the computer is properly programmed, it may also be used as a server.

[0067] Figure 4 This is a block diagram illustrating an exemplary processing device according to some embodiments of this application. For example... Figure 4As shown, the processing device 140 may include an acquisition module 410, a mask determination module 420, and an image registration module 430. The modules may be all or part of the hardware circuitry of the processing device 140. These modules may also be implemented as an application program or instruction set read and executed by the processing device 140. Furthermore, the modules may be any combination of hardware circuitry and application programs / instructions. For example, when the processing device 140 is executing an application program / instruction set, the module may be part of the processing device 140.

[0068] The acquisition module 410 can be configured to acquire information and / or data from one or more components of the medical system 100. For example, the acquisition module 410 can acquire a reference image (e.g., DRR) and a target image (e.g., an X-ray image) of an object, wherein the reference image and the target image are generated at different time points.

[0069] The mask determination module 420 can be configured to determine a registration mask associated with the ROI based on a reference image. For example, the mask determination module 420 can generate an intermediate image by removing image information other than the ROI from the reference image, and then perform a filtering operation on the intermediate image. The mask determination module 420 can determine the registration mask associated with the ROI based on the filtered intermediate image. As another example, the mask determination module 420 can determine a preliminary mask including the ROI based on the reference image. The mask determination module 420 can identify one or more interfering structures in the ROI that are at least partially located in the reference image. The mask determination module 420 can determine an auxiliary mask including one or more interfering structures, and then determine the registration mask associated with the ROI based on the preliminary mask and the auxiliary mask.

[0070] Image registration module 430 can be configured to perform image registration on a reference image and a target image based on a registration mask. In some embodiments, image registration module 430 can determine a mask reference image by applying a registration mask to the reference image, and determine a mask target image by applying a registration mask to the target image. Image registration module 430 can determine one or more offset parameters based on the mask reference image and the mask target image. For example, for each of one or more target regions in the mask target image, image registration module 430 can determine a similarity value between the target region and the corresponding reference region in the mask reference image. Image registration module 430 can determine one or more offset parameters based on one or more similarity values ​​corresponding to one or more target regions.

[0071] The modules in processing device 140 can be interconnected or communicate with each other via wired or wireless connections. Wired connections may include metal cables, optical fibers, hybrid cables, etc., or any combination thereof. Wireless connections may include local area networks (LANs), wide area networks (WANs), Bluetooth, ZigBee, near field communication (NFC), etc., or any combination thereof. Two or more modules may be combined into a single module. Any of the modules may be divided into two or more units. For example, acquisition module 410 may be divided into two units, one for acquiring a reference image of the object and the other for acquiring a target image of the object. As another example, processing device 140 may also include a control module configured to generate control signals for one or more components in medical system 100.

[0072] Figure 5 This is a flowchart illustrating an exemplary process for image registration according to some embodiments of this application. In some embodiments, process 500 may be implemented as a set of instructions (e.g., an application program) stored in storage device 150, memory 220, or memory 390. Processing device 140, processor 210, and / or CPU 340 may execute the instructions, and when the instructions are executed, processing device 140, processor 210, and / or CPU 340 may be configured to perform process 500. The operation of the process shown below is for illustrative purposes only. In some embodiments, process 500 may be performed with one or more additional operations not described and / or without one or more of the operations discussed. Furthermore, Figure 5 The order of operations of process 500 shown and described below is not intended to be restrictive.

[0073] In 510, the processing device 140 (e.g., acquisition module 410) can acquire a reference image of the object.

[0074] The reference image may include the target volume of an object. For example, the object may be a biological object (e.g., a patient, an animal) or a non-biological object (e.g., a phantom). Therefore, the target volume of the object may include an organ, tissue, or a specific portion of the object. By way of example only, the target volume may include a tumor, lesion, etc. In some embodiments, the target volume may include the total target volume (GTV), clinical target volume (CTV), internal target volume (ITV), planning target volume (PTV), treatment volume (TV), irradiation volume (IV), etc., or combinations thereof. In some embodiments, the target volume may be a 2D region, a 3D region, a 4D region, etc.

[0075] In some embodiments, the processing device 140 may acquire reference images from the medical device 110 (e.g., imaging device 111), storage device 150, or any other external device. For example, the imaging device 111 may send reference imaging data (e.g., projection data) of the acquired object to the storage device 150 for storage. The processing device 140 may acquire the reference imaging data from the storage device 150 and generate a reference image based on the reference imaging data. As another example, the processing device 140 may directly acquire the reference image from the storage device 150.

[0076] In some embodiments, the reference image may be a two-dimensional (2D) image, a three-dimensional (3D) image, a four-dimensional (4D) image (e.g., a time-series 3D image), etc. In some embodiments, the reference image may be a planning image related to radiotherapy. For example, the reference image may be a 3D scan image of the object determined based on scan data of the object acquired prior to radiotherapy (e.g., a 3D CT image). As another example, the reference image may be a digitally reconstructed radiograph (DRR) generated from or determined based on the object's scan data (e.g., a DRR generated by projecting a 3D scan image onto an image plane).

[0077] In 520, processing device 140 (e.g., mask determination module 420) can determine a registration mask associated with a region of interest (ROI) based on a reference image. The ROI may include at least a portion of a target volume. As used herein, the registration mask can be used to determine a registration processing region (e.g., the region corresponding to the ROI) in the image to be registered during image registration.

[0078] In some embodiments, the ROI can be determined based on the default settings of the medical system 100, or by a user (e.g., an operator) through an interface such as terminal device 130 or processing device 140. For example, a user can define a contour (e.g., a rectangle) surrounding at least a portion of the target volume via terminal device 130. Processing device 140 can then determine the region corresponding to this contour as the ROI. As another example, processing device 140 can directly determine the region comprising at least a portion of the target volume as the ROI.

[0079] In some embodiments, the registration mask can be considered a "registration mask image" which includes pixels (or voxels) having specified values ​​(e.g., 0 or 1). For example, the registration mask can be a binary image including pixels (or voxels) corresponding to ROIs with a value of "1" and pixels (or voxels) corresponding to the outer regions of ROIs with a value of "0".

[0080] In some embodiments, the dimensions of the reference image and the registration mask image may be the same or different. For example, both the dimensions of the reference image and the registration mask image may be two-dimensional. As another example, the dimensions of the reference image may be 3D and the dimensions of the registration mask image may be 2D or 3D. For example, the processing device 140 may generate a 3D registration mask image based on a 3D reference image and / or generate a 2D registration mask by projecting a 3D registration mask onto a predetermined plane.

[0081] In some embodiments, the processing device 140 may determine a registration mask based on one or more structures (e.g., organs, tissues) in a reference image, wherein these structures may be identified by the processing device 140 or may be pre-segmented (e.g., these structures may be pre-segmented and included in a radiotherapy treatment plan). For example, the processing device 140 may classify one or more structures into an inclusion set or an exclusion set. The inclusion set may include structures located within a Region of Interest (ROI) (e.g., target volume); the exclusion set may include structures located outside the ROI and / or moving or positionally variable structures located or partially within the ROI. Furthermore, the processing device 140 may determine the registration mask by setting the values ​​of pixels (or voxels) corresponding to structures in the inclusion set to 1 and setting the values ​​of pixels (or voxels) corresponding to structures in the exclusion set to 0 or values ​​close to 0. In some embodiments, structures in the inclusion set may also be referred to as target structures, and structures in the exclusion set may also be referred to as interfering structures. In some embodiments, the identified structures may have extended spatial margins. For example, moving or positionally variable structures may have extended spatial margins to accommodate possible motion.

[0082] In some embodiments, the processing device 140 can generate an intermediate image by removing image information other than the Region of Interest (ROI) from a reference image, and determine a registration mask associated with the ROI based on the intermediate image. For example, the processing device 140 can determine the intermediate image by setting the values ​​of pixels (or voxels) outside the ROI in the reference image to 0 or values ​​close to 0. Furthermore, the processing device 140 can determine the registration mask by setting the values ​​of pixels (or voxels) in the ROI in the intermediate image to 1.

[0083] In some embodiments, the processing device 140 may determine a registration mask based on a preliminary mask and an auxiliary mask. The preliminary mask may be a mask that includes the Region of Interest (ROI), and the auxiliary mask may be a mask that includes one or more interfering structures (e.g., musculoskeletal organs, musculoskeletal tissues) that are at least partially located within the ROI. Further description of the determination of the registration mask can be found elsewhere in this application (e.g., Figure 6 (and its description).

[0084] In 530, processing device 140 (e.g., acquisition module 410) can acquire a target image of the object. The target image may include the target volume. Each pixel (or voxel) in the target image may correspond to a pixel (or voxel) in a reference image. As used herein, two corresponding pixels (or voxels) in the reference image and the target image may correspond to the same or similar physical portions or locations of the object.

[0085] In some embodiments, the processing device 140 may acquire a target image from the medical device 110 (e.g., imaging device 111) or the storage device 150. For example, the imaging device 111 may send the acquired target imaging data (e.g., projection data) of the object to the storage device 150 for storage. The processing device 140 may obtain target image data from the storage device 150 and generate a target image based on the target image data. Alternatively, the processing device 140 may directly obtain the target image from the storage device 150.

[0086] In some embodiments, similar to a reference image, a target image can be a 2D image, a 3D image, a 4D image, etc. In some embodiments, a target image can be a measurement image taken during radiotherapy. For example, a target image can be a 3D scan image (e.g., a 3D CT image) of an object determined based on scan data of the object acquired during radiotherapy. As another example, a target image can be a two-dimensional scan image (e.g., an X-ray image) of an object determined based on scan data of the object acquired during radiotherapy.

[0087] In some embodiments, the target image and the reference image can be generated at different points in time. For example, as described above, a reference image (e.g., a planning image) can be generated before radiotherapy, and a target image (e.g., an X-ray image) can be generated during radiotherapy. Alternatively, the reference image and the target image can be generated at different points in time before radiotherapy; for example, the reference image may be generated earlier than the target image. In some embodiments, the target image and the reference image can be generated based on scan data acquired by different imaging devices, i.e., the modalities of the reference image and the target image can be different. For example, the reference image can be a CT image generated from a CT scan prior to radiotherapy; the target image can be an X-ray image generated from an X-ray scan during radiotherapy. Alternatively, the reference image can be a PET image generated from a PET scan; the target image can be an MRI image generated from an MRI scan. Or, the modalities of the reference image and the target image can be different or the same. For example, the reference image and the target image can both be CT images, acquired by the same CT device before and during radiotherapy, respectively.

[0088] In some embodiments, the target image and the reference image may correspond to the same respiratory phase of the object. For example, the respiratory cycle of the object may include an inspiratory phase, a breath-holding phase (i.e., the phase of holding one's breath), and an expiratory phase. Therefore, both the target image and the reference image may correspond to the breath-holding phase. In some embodiments, the reference image and the target image may correspond to the same interval of the respiratory cycle. For example, both the reference image and the target image may be the average of at least two images acquired within a predetermined interval of the respiratory cycle.

[0089] In 540, the processing device 140 (e.g., image registration module 430) can perform image registration on the reference image and the target image based on the registration mask.

[0090] In some embodiments, image registration may include 3D-3D registration, 3D-2D registration, 2D-2D registration, or combinations thereof. In some embodiments, when the dimensions of the reference image, the target image, and the registration mask are the same, the processing device 140 may directly apply the registration mask to the reference image and the target image to perform image registration.

[0091] In some embodiments, when the dimensions of the reference image, target image, and / or registration mask are different, the processing device 140 can process the reference image, target image, and / or registration mask to have the same dimensions. The processing device 140 can perform image registration based on the processed image. For example, when the reference image and registration mask are 3D images, and the target image is a 2D image, the processing device 140 can generate a 2D reference image and a 2D registration mask by projecting the 3D reference image and the 3D registration mask onto a predetermined plane. The processing device 140 can perform image registration on the 2D reference image and the 2D target image based on the 2D registration mask. As another example, when the reference image and registration mask are 3D images, and the target image is a 2D image, the processing device 140 can generate a 3D target image by reconstructing the 2D target image. The processing device 140 can perform image registration on the 3D reference image and the 3D target image based on the 3D registration mask. As a further example, when the reference image and registration mask are 2D images and the target image is a 3D image, the processing device 140 can generate a 2D target image by projecting the 3D target image onto a predetermined plane. The processing device 140 can perform image registration on the 2D reference image and the 2D target image based on the 2D registration mask.

[0092] In some embodiments, the processing device 140 may use image registration techniques to perform image registration. In some embodiments, image registration techniques may include grayscale-based techniques, transform domain-based techniques, feature-based techniques, etc., or any combination thereof. In some embodiments, the processing device 140 may determine a mask reference image and a mask target image by applying registration masks to a reference image and a target image, respectively. Furthermore, the processing device 140 may perform image registration on the mask reference image and the mask target image. In some embodiments, the processing device 140 may determine one or more offset parameters based on the mask reference image and the mask target image. Further description of image registration between a reference image and a target image based on a registration mask can be found elsewhere in this application (e.g., Figure 7 (and its description).

[0093] It should be noted that the above description of process 500 is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various changes and modifications based on the description in this application. However, these changes and modifications do not depart from the scope of this application. In some embodiments, one or more operations may be omitted and / or one or more additional operations may be added. For example, process 500 may also include an operation in which the processing device 140 drives the worktable 113 of the medical system 100 to move according to one or more offset parameters. As another example, process 500 may further include an operation in which the processing device 140 sends one or more offset parameters to the display of a user's (e.g., a doctor's) terminal device (e.g., terminal device 130). The user can manually move the worktable 113 of the medical system 100 according to one or more movement parameters.

[0094] Figure 6 This is a flowchart illustrating an exemplary process for determining a registration mask according to some embodiments of this application. In some embodiments, process 600 may be implemented as a set of instructions (e.g., an application program) stored in storage device 150, storage device 220, or storage device 390. Processing device 140, processor 210, and / or CPU 340 may execute the instructions, and when the instructions are executed, processing device 140, processor 210, and / or CPU 340 may be configured to perform process 600. The operation of the process shown below is for illustrative purposes only. In some embodiments, process 600 may be performed with one or more additional operations not described and / or without one or more of the operations discussed. Furthermore, Figure 6 The order of operations of process 600 shown and described below is not intended to be limiting. In some embodiments, operation 520 may be performed based on process 600.

[0095] In 610, processing device 140 (e.g., mask determination module 420) can generate an intermediate image by removing image information other than the ROI from the reference image. For example, processing device 140 can set the values ​​of reference pixels (or voxels) outside the ROI of the reference image to 0 or values ​​close to 0. Therefore, the intermediate image only includes image information of the ROI. Alternatively, special values, such as NaN (not a number, e.g., in the IEEE 754 floating-point encoding standard), can be assigned to pixel (or voxel) values ​​that are excluded as information that can be used for registration to indicate that these values ​​are not considered.

[0096] In 620, processing device 140 (e.g., mask determination module 420) can determine a preliminary mask containing the ROI based on the intermediate image. As used herein, the preliminary mask can be used to identify or locate the ROI. For example, processing device 140 can determine the preliminary mask by setting the value of the pixel (or voxel) corresponding to the ROI in the intermediate image to 1. Therefore, the preliminary mask can be used to extract or distinguish image information of the ROI from the reference image.

[0097] In some embodiments, the dimensions of the initial mask may be the same as the dimensions of the intermediate image (or reference image). For example, when the intermediate image (or reference image) is a 3D image, the initial mask may be a 3D mask. In some embodiments, the dimensions of the initial mask may be different from the dimensions of the intermediate image (or reference image).

[0098] In some embodiments, processing device 140 may perform preprocessing operations on intermediate images before determining a preliminary mask. In some embodiments, preprocessing operations may include filtering operations, smoothing operations, denoising operations, etc., or any combination thereof. Processing device 140 may determine a preliminary mask based on the processed intermediate image. For example, processing device 140 may determine a smoothed intermediate image by performing a smoothing operation on pixels (or voxels) in the intermediate image that have values ​​exceeding a threshold. Processing device 140 may determine a preliminary mask based on the smoothed intermediate image. The threshold may be set by the user or according to the default settings of medical system 100, or may be adjusted under different circumstances. As another example, processing device 140 may determine a filtered intermediate image by performing a filtering operation on the intermediate image, such that artifacts or noise are reduced. Processing device 140 may determine a preliminary mask based on the filtered intermediate image.

[0099] In 630, the processing device 140 (e.g., mask determination module 420) can identify one or more interfering structures that are at least partially located in the ROI in the reference image.

[0100] As used herein, an interfering structure refers to a structure that may affect the registration accuracy and / or stability between a reference image and a target image. For example, a moving structure or a structure with variable position due to physiological motion (e.g., respiratory motion, cardiac motion, arterial pulsation) can be considered an interfering structure, since registration between medical images is often sensitive to the motion of organs or tissues. In some embodiments, the processing device 140 may extend the spatial margin of the interfering structure to accommodate possible motion of the interfering structure.

[0101] In some embodiments, one or more interfering structures may include one or more musculoskeletal organs or tissues, such as the heart, lungs (e.g., left lung, right lung), diaphragm, bladder, rectum, etc., or a portion thereof. In some embodiments, the interfering structures may differ for different therapeutic purposes. For example, for chest treatment, one or more interfering structures may include the left lung, right lung, heart, diaphragm, etc., or any combination thereof. As another example, for abdominal treatment, one or more interfering structures may include a musculoskeletal portion of the digestive tract having variable contents (e.g., rectum) and / or containing variable amounts of gas. As a further example, for abdominal treatment, one or more interfering structures may include a bladder containing variable amounts of fluid.

[0102] In some embodiments, when the reference image is a DRR generated by projecting a 3D scan image onto an image plane, one or more interfering structures can be identified in the 3D scan image other than the reference image and can be projected onto the image plane using projection geometry (possibly similar to the projection geometry used to generate the reference image (i.e., the DRR image)). In some embodiments, for the projection operation, the processing device 140 can perform one or more operations, including using the same energy spectrum as the actual imaging beam, modeling beam scattering or alternatively compensating for DRR for beam scattering, modeling the detector response function (e.g., energy deposition response, spatial response) of the detector used to acquire the DRR, or any combination thereof.

[0103] In some embodiments, processing device 140 can identify one or more interfering structures based on structure type and / or structure tag (or structure identifier). For example, processing device 140 can identify one or more interfering structures based on multiple structure types and / or structure tags stored in a Radiation Therapy Structure Set (RTSS). In other embodiments, the reference image can be generated according to a 4D imaging mode (including time information during imaging) and / or a cinematic imaging mode, i.e., the reference image (or reference cinema image) can reflect time information; therefore, processing device 140 can identify one or more interfering structures based on the time information associated with the reference image. For example, moving structures or variable-position structures can appear in different time periods and can be identified based on the time information associated with the reference image (or reference cinema image). Specifically, moving structures or variable-position structures can be identified based on information about how each structure changes over time. For example, processing device 140 can determine that a structure whose features (e.g., position, shape, size, etc.) change over time is a moving structure or a variable-position structure.

[0104] In some embodiments, the processing device 140 may identify one or more interfering structures from a target image generated during object processing in a manner similar to identifying interfering structures from a reference image. For example, the target image may be generated according to a 4D imaging scheme (including temporal information during imaging) and / or a cinematic imaging scheme. The processing device 140 may identify one or more interfering structures based on temporal information associated with the target image.

[0105] In 640, processing device 140 (e.g., mask determination module 420) can determine an auxiliary mask that includes one or more interfering structures. As used herein, similar to a preliminary mask, the auxiliary mask can be used to identify or locate one or more interfering structures. For example, processing device 140 can determine the auxiliary mask by setting the values ​​of pixels (or voxels) in one or more interfering structures to 0 or near-0, and setting the values ​​of pixels (or voxels) outside one or more interfering structures to 1. As another example, processing device 140 can determine the auxiliary mask by setting the values ​​of pixels (or voxels) in one or more interfering structures to 1 and setting the values ​​of pixels (or voxels) outside one or more interfering structures to 0 or near-0.

[0106] In 650, processing device 140 (e.g., mask determination module 420) can determine the registration mask associated with the ROI based on a preliminary mask and an auxiliary mask.

[0107] In some embodiments, the processing device 140 can determine the registration mask by combining a preliminary mask and an auxiliary mask. For example, if the values ​​of one or more pixels (or voxels) in the interfering structures are set to 0 in the auxiliary mask, the processing device 140 can determine the registration mask by multiplying the preliminary mask by the auxiliary mask. As another example, if the values ​​of one or more pixels (or voxels) in the interfering structures are set to 1 in the auxiliary mask, the processing device 140 can determine the registration mask by subtracting the auxiliary mask from the preliminary mask. Accordingly, by considering only structures in the ROI other than interfering structures during image registration, image registration accuracy can be improved, and the influence of interfering structures can be removed or reduced.

[0108] It should be noted that the above description of process 600 is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various changes and modifications based on the description in this application; however, these changes and modifications do not depart from the scope of this application. In some embodiments, one or more operations may be omitted and / or one or more additional operations may be added. For example, operations 630 and 640 may be combined into a single operation, wherein the processing device 140 can identify one or more interference structures and determine an auxiliary mask based on one or more interference structures.

[0109] Figure 7 This is a flowchart illustrating an exemplary process for registering a reference image to a target image based on a registration mask, according to some embodiments of this application. In some embodiments, process 700 may be implemented as a set of instructions (e.g., an application program) stored in storage device 150, storage device 220, or storage device 390. Processing device 140, processor 210, and / or CPU 340 may execute the instructions, and when executing the instructions, processing device 140, processor 210, and / or CPU 340 may be configured to perform process 700. The operation of the process shown below is for illustrative purposes only. In some embodiments, process 700 may be performed with one or more additional operations not described and / or without one or more of the operations discussed. Furthermore, Figure 7 The order of operations of process 700 shown and described below is not intended to be limiting. In some embodiments, operation 540 may be performed based on process 700.

[0110] In 710, processing device 140 (e.g., image registration module 430) can determine a mask reference image by applying a registration mask to a reference image. For example, processing device 140 can determine the mask reference image by multiplying the registration mask by the reference image. Specifically, processing device 140 can determine the mask reference image by multiplying the value of each pixel (or voxel) in the reference image by the value of the corresponding pixel (or voxel) in the registration mask. As described elsewhere in this application, since the registration mask only contains information corresponding to the ROI and not information about the interfering structure, the mask reference image can contain only image information of the ROI and not information about the interfering structure.

[0111] In 720, processing device 140 (e.g., image registration module 430) can determine a mask target image by applying a registration mask to the target image. Processing device 140 can determine the mask target image in a manner similar to that used to determine a mask reference image.

[0112] In step 730, for each of one or more target regions in the mask target image, the processing device 140 (e.g., image registration module 430) can determine a similarity value between the target region and a corresponding reference region in the mask reference image. The similarity value can indicate the spatial transformation between the target region in the mask target image and the corresponding reference region in the mask reference image.

[0113] In some embodiments, the processing device 140 can determine a similarity value between a target region and a corresponding reference region based on features (e.g., grayscale value, size, contrast, resolution) of the target region and the corresponding reference region. For example, assuming that both the target region and the reference region correspond to a pixel (or voxel), the processing device 140 can determine a similarity value based on the difference between the features (e.g., grayscale value) of the target pixel (or voxel) in the mask target image and the features (e.g., grayscale value) of the reference pixel (or voxel) in the mask reference image. As another example, assuming that both the target region and the reference region correspond to a "region," the processing device 140 can determine a weighted target feature of the target region by weighting the pixels (or voxels) in the target region and a weighted reference feature of the reference region by weighting the pixels (or voxels) in the reference region. Then, the processing device 140 can determine a similarity value based on the difference between the weighted target feature of the target region and the weighted reference feature of the reference region.

[0114] In 740, processing device 140 (e.g., image registration module 430) can determine one or more offset parameters based on one or more similarity values ​​corresponding to one or more target regions. As used herein, one or more offset parameters can be used to match a mask target image (or target image) with a mask reference image (or reference image).

[0115] For example, when the reference image is a planning image (e.g., CT DRR) and the target image is a treatment image during radiotherapy (e.g., an X-ray image), the positions of one or more components of the medical system 100 (e.g., the workbench 113, the gantry of the treatment device 112) can be adjusted based on one or more offset parameters (e.g., translation parameters, rotation parameters) so that radiotherapy can be accurately delivered to the target volume of the object. As another example, when the reference image is a PET image and the target image is an MRI image, the processing device 140 can determine a fused image by fusing the PET and MRI images based on one or more offset parameters, thereby providing object-related anatomical and / or functional information through the fused image to improve the accuracy of diagnostic and / or therapeutic purposes.

[0116] In some embodiments, the processing device 140 may determine a similarity map based on similarity values. The processing device 140 may determine one or more offset parameters based on the similarity map.

[0117] In some embodiments, the processing device 140 may determine one or more offset parameters based on a registration cost function, which may be a similarity measure (e.g., mean squared error (MSE), mutual information (MI), or normalized cross-correlation (NCC) measure) representing the similarity between the target mask image and the reference mask image. For example, the processing device 140 may determine one or more initial offset parameters based on one or more initial similarity values ​​corresponding to one or more target regions. The processing device 140 may then optimize the registration cost function by iteratively updating one or more initial offset parameters using an optimization algorithm until the updated one or more similarity values ​​satisfy a preset condition. In some embodiments, the optimization algorithm may include Powell's optimization algorithm (POA), simulated annealing algorithm, genetic optimization algorithm, exhaustive search algorithm, regular gradient descent algorithm (RSGDA), fruit fly optimization algorithm (FOA), bacterial chemotactic fruit fly optimization algorithm (BCFOA), etc., or any combination thereof.

[0118] It should be noted that the above description of process 700 is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various changes or modifications based on the teachings of this application. However, these changes and modifications do not depart from the scope of this application.

[0119] Example

[0120] The following examples are provided for illustrative purposes and are not intended to limit the scope of this application.

[0121] Figure 8AAn exemplary image registration process based on a registration mask between a planning image and a treatment image during treatment of a chest region is shown according to some embodiments of this application. Figure 8A The image in the image corresponds to the zero-degree rack position. Figure 8B An exemplary image registration process based on a registration mask between a planning image and a treatment image during treatment of a chest region is shown according to some embodiments of this application. Figure 8B The image corresponds to a 90-degree gantry position. During chest region treatment, the left lung and pulmonary diaphragm can be considered interfering structures because they move with the patient's breathing. Therefore, as combined with... Figure 6 As described, a registration mask can be generated corresponding to the ROI of the chest region after removing interfering structures.

[0122] like Figure 8A As shown, Image 1a is the patient's first DRR (i.e., reference image) generated at any time prior to treatment of the chest region. The first DRR can be generated based on a geometric calibration model of the beam spectrum, detector response function, and linear accelerator imaging system geometric calibration. The first DRR can be used as a first planning image. Image 2a is a first X-ray image (i.e., target image) measured in real time during treatment of the chest region. The first X-ray image can be used as a first treatment image. Image 3a is a first visualization image determined by applying a first registration mask to the first DRR and the first X-ray image. Image 4a is a first registration image representing the final registration result of the first DRR and the first X-ray image, i.e., the first X-ray image shifted based on one or more offset parameters.

[0123] Furthermore, bounding box 810 in image 1a represents the default ROI of the first DRR, and bounding box 815 in image 2a represents the default ROI in the first X-ray image; black area 820 refers to the first auxiliary mask corresponding to the interfering structure (e.g., the left lung). Area 830 refers to the first preliminary mask including the ROI. Therefore, the first registration mask is a mask determined by multiplying the first preliminary mask by the first auxiliary mask. That is, the interfering structure is excluded from the ROI during the registration process. Markers 842, 844, 846, and 848 are features identified manually or automatically for visual performance evaluation.

[0124] Figure 8B The image in the image is similar to Figure 8AThe images are shown in the image. For example, image 1b is a second DRR (Digital Register) of the patient generated at any time before treatment of the chest region. The second DRR can be used as a second planning image. Image 2b is a second X-ray image of the patient measured in real time during treatment of the chest region. The second X-ray image can be used as a second treatment image. Image 3b is a second visualization image determined by applying a second registration mask to the second DRR and the second X-ray image. Image 4b ​​is a second image representing the final registration result of the second DRR and the second X-ray image, i.e., the second X-ray image is shifted based on one or more offset parameters.

[0125] Furthermore, bounding box 850 in image 1b refers to the default ROI of the second DRR, and bounding box 855 in image 2b refers to the default ROI in the second X-ray image; the black area 860 refers to the second auxiliary mask corresponding to the interfering structure (e.g., the left lung). Area 870 refers to the second preliminary mask that includes the ROI. Therefore, the second registration mask is a mask determined by multiplying the second preliminary mask by the second auxiliary mask. That is, the interfering structure is excluded from the ROI during the registration process. Markers 882, 884, 886, and 888 are also features identified manually or automatically for visual performance evaluation.

[0126] In some embodiments, during the image registration process, one or more offset parameters can be determined using the mean squared error (MSE) cost function. In some embodiments, including a mask (which may also be considered a "registration mask") can be expressed as the following equation (1):

[0127] Iv=Rv\Xv, (1)

[0128] Among them, I v Indicates the inclusion of a mask; R v Represents the ROI (which can also be considered a "preliminary mask"); X v This indicates the exclusion mask (which can also be considered an "auxiliary mask").

[0129] In some embodiments, in a continuous spatial domain, the first mean square error (MSE) cost function can be expressed as the following equation (2):

[0130]

[0131] Where s represents the offset parameter, and the vector of offset parameters can be defined as s = [δ x δ y δ z ] T ;u v Indicates image coordinates; M v (u v) represents an X-ray image (e.g., a first X-ray image or a second X-ray image); M v (u v -s) represents a shifted (or transformed) X-ray image (e.g., a first shifted X-ray image or a second shifted X-ray image) determined based on an offset parameter; F v (u v ) indicates a DRR (e.g., a first DRR or a second DRR). Because Figure 8A and 8B The images in the image share a single axis (y), therefore the image coordinates corresponding to the 0-degree rack position can be defined as u. 0 degree =[xy] T Furthermore, the image coordinates corresponding to the 90-degree rack position can be defined as u. 90 degree =[zy] T .

[0132] The offset parameters can then be determined by optimizing (e.g., minimizing) the first MSE cost function. In some embodiments, decoupling the estimation may be useful, at least at the start of the optimization, in order to provide faster convergence to the approximate solution.

[0133] In some embodiments, in the discrete spatial domain, the second MSE cost function can be expressed as the following equation (3):

[0134]

[0135] Where s represents the offset parameter, and the vector of offset parameters can be defined as follows: in m and n represent the image pixel indices; and This represents the corresponding pixel offset, which can be an integer or a fraction of pixels. In the latter case, interpolation can be used to calculate the second MSE cost function. Alternatively, if subpixel resolution is required, the image can be upsampled before offset optimization.

[0136] exist Figure 8A and 8B During image registration, an exhaustive search can be used to evaluate the cost function over all pixel offsets within a ±3.5 cm range. The optimal offset parameter was found to be s = [1.95 -2.93 0.977] mm. It should be noted that these examples are for illustrative purposes only, illustrating linear shift (translation) registration transformations. It should be understood that affine and deformable registration (elastic) transformations can be applied similarly.

[0137] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0138] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0139] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of software and hardware, all of which are generally referred to herein as “modules,” “units,” “components,” “devices,” or “systems.” Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0140] A computer-readable signal medium may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. Such propagated signals can take many forms, including electromagnetic, optical, and any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device, or apparatus to enable communication, propagation, or transmission of a program for use. The program code located on the computer-readable signal medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, and any combination of the above.

[0141] Computer program code for performing various aspects of this application can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc.; traditional procedural programming languages ​​such as the "C" programming language, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. The program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., through the network of an internet service provider) or in a cloud computing environment, or provided as a service, such as a software service (SaaS).

[0142] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although the foregoing disclosure has discussed some currently considered useful embodiments of the invention through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, while the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0143] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, the method of the present application should not be construed as reflecting an intention that the claimed object to be scanned requires more features than expressly recited in each claim. In fact, the embodiments have fewer features than all the features of the single embodiments disclosed above.

Claims

1. An image registration system, comprising: at least one storage device storing executable instructions, and at least one processor in communication with the at least one storage device, wherein when executing the executable instructions, the at least one processor causes the system to perform operations comprising: acquiring a reference image of the scanned subject, the reference image including a target volume to which radiation therapy is to be delivered; determining, based on the reference image, a registration mask associated with a region of interest (ROI), the ROI including at least a portion of the target volume and having clinical relevance to the target volume, the registration mask being used to exclude at least part of interfering structures during image registration, the at least part of interfering structures including moving or positionally variable structures located or partially located in the ROI; acquiring a target image of the scanned object, wherein the target image includes the target volume; and Based on the registration mask, image registration is performed on the reference image and the target image.

2. The system according to claim 1, characterized in that The determining, based on the reference image, a registration mask associated with the ROI comprises: removing image information outside the ROI from the reference image to generate an intermediate image; and Based on the intermediate image, the registration mask associated with the ROI is determined.

3. The system according to claim 2, characterized in that The determining, based on the intermediate image, the registration mask associated with the ROI comprises: performing a filtering operation on the intermediate image; and Based on the filtered intermediate image, the registration mask associated with the ROI is determined.

4. The system according to claim 1, characterized in that The determining, based on the reference image, a registration mask associated with the ROI comprises: Based on the reference image, determining a preliminary mask including the ROI; identifying one or more interfering structures in the ROI at least partially located in the reference image; determining an auxiliary mask including the one or more interference structures; and The registration mask associated with the ROI is determined based on the preliminary mask and the auxiliary mask.

5. The system according to claim 4, characterized in that The identifying one or more interfering structures at least partially located in the ROI in the reference image comprises: The one or more interfering structures are automatically segmented from the reference image according to a preset rule.

6. The system according to claim 5, characterized in that The preset rule includes at least one of a structure type, a structure label or time information associated with the reference image.

7. The system according to claim 4, characterized in that The one or more interfering structures include at least one of a moving organ or a moving tissue.

8. The system according to claim 7, characterized in that The movement organ or the movement tissue is determined based on a reference movie image.

9. The system according to claim 7, characterized in that The movement organ or the movement tissue is determined based on a target movie image.

10. The system according to claim 7, characterized in that The moving organ or the moving tissue is determined according to the movement characteristics of the organ or tissue identified in the target image or the reference image.

11. The system according to claim 4, characterized in that The one or more interfering structures include at least one of a heart, a lung, a diaphragm, a bladder, or a rectum.

12. The system of claim 1, wherein: The performing image registration on the reference image and the target image based on the registration mask comprises: determining a mask reference image by applying the registration mask on the reference image; determining a mask target image by applying the registration mask on the target image; and Based on the mask reference image and the mask target image, one or more offset parameters are determined.

13. The system of claim 12, wherein: The determining one or more offset parameters based on the mask reference image and the mask target image comprises: For each of the one or more target regions in the mask target image, determining a similarity value between the target region and a corresponding reference region in the mask reference image; and The one or more shift parameters are determined based on the one or more similarity values ​​corresponding to the one or more target regions.

14. The system of claim 1, wherein: The reference image and the target image are acquired in the same respiratory phase.

15. The system of claim 1, wherein: The reference image is a planning image associated with the treatment; and The target image is a treatment image before or during treatment.

16. The system of claim 15, wherein: The planning image is a digitally reconstructed radiograph (DRR); and The treatment image is an X-ray image.

17. The system of claim 1, wherein: Each of the reference image and the target image is a 3D image or a 4D image.

18. An image registration method implemented on a computing device having at least one processor and at least one storage device, the method comprising: acquiring a reference image of the scanned subject, the reference image including a target volume to which radiation therapy is to be delivered; determining, based on the reference image, a registration mask associated with a region of interest (ROI), the ROI including at least a portion of the target volume and having clinical relevance to the target volume, the registration mask being used to exclude at least part of interfering structures during image registration, the at least part of interfering structures including moving or positionally variable structures located or partially located in the ROI; Acquire a target image of the scanned object, wherein the target image includes the target volume; as well as Based on the registration mask, image registration is performed on the reference image and the target image.

19. The method according to claim 18, characterized in that The performing image registration on the reference image and the target image based on the registration mask comprises: determining a mask reference image by applying the registration mask on the reference image; determining a mask target image by applying the registration mask on the target image; and Based on the mask reference image and the mask target image, one or more offset parameters are determined.

20. A non-transitory computer readable medium comprising at least one set of instructions for image registration, characterized in that: When executed by at least one processor of a computing device, the at least one set of instructions instructs the at least one processor to perform operations comprising: acquiring a reference image of the scanned subject, the reference image including a target volume to which radiation therapy is to be delivered; determining, based on the reference image, a registration mask associated with a region of interest (ROI), the ROI including at least a portion of the target volume and having clinical relevance to the target volume, the registration mask being used to exclude at least part of interfering structures during image registration, the at least part of interfering structures including moving or positionally variable structures located or partially located in the ROI; Acquire a target image of the scanned object, wherein the target image includes the target volume; as well as Based on the registration mask, image registration is performed on the reference image and the target image.

Images