Method and system for generating region of interest

Abstract

The embodiment of the invention discloses a method for generating a region of interest, and the method comprises the steps: obtaining a three-dimensional image of an object, and enabling the three-dimensional image to comprise a three-dimensional contour of the object; determining a two-dimensional contour of the object in a preset direction based on the three-dimensional image; obtaining a two-dimensional medical image of the object in the preset direction; generating a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour; and determining a region of interest from the two-dimensional image.

Description

[0001] This application is a divisional application of the invention patent application with application date of October 28, 2021, application number 202111261713.X, and invention name “A method and system for generating a region of interest”. Technical Field

[0002] The present application relates to the field of image processing, and in particular to a method and system for generating a region of interest. Background Art

[0003] Before radiotherapy is performed on a patient, it is usually necessary to determine the region of interest (ROI). The ROI can be used to plan the specific content of the subsequent treatment plan, such as the radiation dose distribution in each region. In order to more intuitively display the patient's treatment object (such as organs, lesions, etc.), some tools can be used to display the image of the treatment object for reference. However, the treatment object is usually presented in the form of a three-dimensional image. However, since three-dimensional images have more complex stereoscopic properties than two-dimensional images, it is often difficult to determine the ROI directly on the three-dimensional image.

[0004] Therefore, it is necessary to propose a method to generate a region of interest to determine the ROI in treatment planning more simply and efficiently. Summary of the invention

[0005] One of the embodiments of the present specification provides a method for generating a region of interest, the method comprising: acquiring a three-dimensional image of an object, the three-dimensional image comprising a three-dimensional contour of the object; determining a two-dimensional contour of the object in a preset direction based on the three-dimensional image; acquiring a two-dimensional medical image of the object in the preset direction; generating a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour; and determining a region of interest from the two-dimensional image.

[0006] One of the embodiments of the present specification also provides a system for generating a region of interest, characterized in that the system includes: a first acquisition module, used to acquire a three-dimensional image of an object, the three-dimensional image including a three-dimensional contour of the object; a first determination module, used to determine a two-dimensional contour of the object in a preset direction based on the three-dimensional image; a second acquisition module, used to acquire a two-dimensional medical image of the object in the preset direction; a two-dimensional image generation module, used to generate a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour; and a second determination module, used to determine a region of interest from the two-dimensional image. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present application will be further described in the form of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0008] Figure 1 is a schematic diagram of an application scenario of a system for generating a region of interest according to some embodiments of this specification;

[0009] Figure 2 is an exemplary module block diagram of a system for generating a region of interest according to some embodiments of the present specification;

[0010] Figure 3 is an exemplary flow chart of a method for generating a region of interest according to some embodiments of this specification;

[0011] Figure 4A is a schematic diagram of a three-dimensional image cutting process according to some embodiments of this specification;

[0012] Figure 4B According to this manual Figure 4A Schematic diagram of the cross section at the D7 level;

[0013] Figure 5 is a schematic diagram of a user interaction interface according to some embodiments of this specification;

[0014] Figure 6 is an exemplary flow chart of determining a two-dimensional contour of an object according to some embodiments of this specification;

[0015] Figure 7 is an exemplary flow chart of determining a region of interest according to some embodiments of the present specification. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, this specification can be implemented without adopting the details. In other cases, in order to avoid unnecessarily blurring the various aspects of this specification, well-known methods, processes, systems, components and / or circuits have been described in general terms at a relatively high level. For those of ordinary skill in the art, various modifications to the embodiments disclosed in this specification are obvious, and the general principles defined in this specification can be applied to other embodiments and application scenarios without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but conforms to the widest scope consistent with the scope of the specification.

[0017] The terms used in this specification are only used to describe specific exemplary embodiments and do not limit the scope of this specification. The singular forms "a", "an" and "the" used in this specification may also include plural forms unless the context clearly indicates an exception. It should also be understood that, as in this specification, the terms "include" and "comprise" only indicate the presence of the features, wholes, steps, operations, components and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, components, parts and / or combinations thereof.

[0018] It should be understood that the "system", "unit", "module", "and / or", "block" used in this specification is a method of distinguishing different components, elements, parts, portions or assemblies at different levels in ascending order. However, another expression that achieves the same purpose can be used to replace the above terms.

[0019] It should be understood that when a unit, engine, module or block is referred to as being "on," "connected to," or "coupled to" another unit, engine, module or block, it may be directly on, connected to, or coupled to other units, engines, modules or blocks, or communicate with other units, engines, modules or blocks, or intervening units, engines, modules or blocks. In this specification, the term "and / or" may include any one or more of the relevant listed items or combinations thereof.

[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the operations may be processed in reverse order or simultaneously. At the same time, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0021] The features and other features, methods of operation, functions of related components and economical structures described in this specification will become more apparent from the following description of the accompanying drawings, which all constitute a part of the specification. This specification provides systems and components for medical imaging and / or medical treatment, such as systems and components for disease diagnosis, treatment or research purposes. In some embodiments, the medical system may include an imaging system. The imaging system may include a single-modality imaging system and / or a multi-modality imaging system. The term "modality" as used herein refers to an imaging or treatment method or technique for collecting, generating, processing and / or analyzing imaging information of an object or treating the object. Single-modality systems may include, for example, computed tomography (CT) systems, X-ray imaging systems, digital radiography (DR) systems, magnetic resonance imaging (MRI) systems, positron emission tomography (PET) systems, single-photon emission computed tomography (SPECT) systems, optical coherence tomography (OCT) systems, ultrasound (US) systems, near infrared spectroscopy (NIRS) systems, etc. or any combination thereof. The multimodal system may include, for example, a positron emission tomography-computed tomography (PET-CT) system, a positron emission tomography-magnetic resonance imaging (PET-MRI) system, a computed tomography-magnetic resonance imaging (CT-MRI) system, a single photon emission computed tomography-magnetic resonance imaging (SPECT-MRI) system, a digital subtraction angiography (DSA)-magnetic resonance imaging (DSA-MRI) system, etc. or any combination thereof. In some embodiments, the computed tomography (CT) system may include a C-arm system using X-rays, a dental CT, or a CT system using other types of rays, etc.

[0022] In some embodiments, the medical system may include a treatment system. The treatment system may include a treatment planning system (TPS), an image guided radiation therapy (IGRT) system, a radiation therapy delivery system, etc. Image guided radiation therapy (IGRT) may include a treatment device and an imaging device. The treatment device may include a linear accelerator, a cyclotron, a synchrotron, etc., which are configured to perform radiation therapy on a subject. The treatment device may include accelerators of various particle types, such as photons, electrons, protons, or heavy ions. The imaging device may include an MRI scanner, a CT scanner (e.g., a cone beam computed tomography (CBCT) scanner), a digital radiology (DR) scanner, an electronic portal imaging device (EPID), etc. The medical system described below is provided for illustrative purposes only and does not limit the scope of this specification.

[0023] In this specification, subject can include organism and / or non-organism. Organism can be human, animal, plant or its specific part, organ and / or tissue. For example, subject can include head, neck, chest, lung, heart, stomach, blood vessel, soft tissue, tumor, nodule etc. or its any combination. In certain embodiments, subject can be the man-made composition of organic and / or inorganic substance with life or without life. In this specification, term " object " or " subject " can be used interchangeably.

[0024] In this specification, a representation of an object in an image (e.g., a patient, a subject, or a portion thereof) may be referred to as an object. For example, a representation of an organ and / or tissue in an image (e.g., a heart, a liver, a lung) may be referred to as an organ or a tissue. An image including a representation of an object may be referred to as an image of the object or an image including the object. An operation performed on a representation of an object in an image may be referred to as an operation on the object. For example, segmentation of a portion of an image including a representation of an organ and / or tissue may be referred to as segmentation of the organ and / or tissue.

[0025] In this specification, the user may include an operator (e.g., a doctor, a pharmacist) who uses the method of the present application, or may include a display object (e.g., a patient, a family member) of the method. In this specification, the term "user" or "operator" may be used interchangeably.

[0026] In radiotherapy planning, in order to observe the condition of the object more intuitively, a medical image sequence of the object (e.g., multiple CT images) can be obtained to determine the density field, thereby generating a three-dimensional image of the object (e.g., a three-dimensional anatomical image of the object). To further perform image-guided radiotherapy (IGRT), an operator (e.g., a practicing radiotherapist) is required to determine a region of interest (ROI). For example, the region of interest may include a lesion and / or a healthy area around the lesion, etc., so as to determine that the radiation dose of each region of interest in the image-guided radiotherapy meets the standard. In this process, the operator can directly operate the three-dimensional image, but the display and interaction mode of the three-dimensional image makes it difficult to perform related operations. For example, when determining the region of interest, the operator may need to make some modifications to the contours of the organ, tissue or lesion in the three-dimensional image (e.g., expand the selection), but the three-dimensional image itself is relatively complex, and it is difficult for the operator to accurately modify the contour directly based on the three-dimensional image.

[0027] Therefore, the present invention proposes to display a three-dimensional image in a two-dimensional form to help users perform operations related to the region of interest. For example, some embodiments of the present specification provide a method and system for generating a region of interest, which can present the contours of each organ, tissue or lesion in a three-dimensional image in a two-dimensional manner by cross-sectional cutting, and combine the two-dimensional contour with the two-dimensional medical image to generate multiple two-dimensional images containing three-dimensional image cross-sectional information and two-dimensional medical images, so as to facilitate relevant operators to determine the region of interest from the two-dimensional image and facilitate users to perform operations related to the region of interest, such as determining the range of the region of interest or setting the radiation dose for the region of interest.

[0028] Figure 1 Schematic diagram of application scenarios of the system for generating a region of interest according to some embodiments of this specification. Figure 1 As shown, the region of interest generation system 100 may include a processing device 110, a terminal device 120, a storage device 130, a network 140, and a medical device 150. The various components in the region of interest generation system 100 may be connected in a variety of ways. For example, the processing device 110 and the terminal device 120 may be connected via the network 140, or may be directly connected (e.g., Figure 1 As another example, the storage device 130 and the processing device 110 may be connected directly or via the network 140. As another example, the terminal device 130 and the processing device 120 may be connected via the network 140 or directly.

[0029] The processing device 110 is the execution subject of the method for generating a region of interest in this specification. For example, the processing device 110 can determine the two-dimensional outline of the object in a preset direction based on the three-dimensional image. For another example, the processing device can generate a two-dimensional image based on the two-dimensional medical image of the object and the two-dimensional outline of the object. In some embodiments, the processing device 110 can be used to generate a region of interest based on the two-dimensional image. In some embodiments, the object may include organs, tissues, or lesions involved in a medical task.

[0030] The terminal device 120 may be a device capable of inputting and outputting data or processing results. In some embodiments, the terminal device 120 may present an area of ​​interest. For example, a two-dimensional image containing an area of ​​interest processed by the processing device 110 may be sent to the terminal device 120 via the network 140, and the terminal device 120 may present the processed two-dimensional image to the user. In some embodiments, the terminal device 120 may include a mobile device 121, a tablet computer 122, a laptop computer 123, or the like, or any combination thereof. To facilitate user observation or operation, the terminal device 120 may be a display device for presenting a three-dimensional image. For example, the terminal device 120 may be a device capable of displaying a three-dimensional image using virtual reality (VR) technology or augmented reality (AR) technology (e.g., VR / AR glasses 124). In some embodiments, the terminal device 120 may implement a user interaction function. For example, an operator may input instructions to the terminal device 120 to edit the area of ​​interest.

[0031] The storage device 130 may store data (e.g., medical images, images to be processed, three-dimensional images, etc.), instructions, and / or any other information. In some embodiments, the storage device 130 may store data obtained from the imaging device, the terminal device 120, and / or the processing device 110. For example, the storage device 130 may store scan data obtained from the imaging device and store it as an image to be processed. In some embodiments, the storage device 130 may store data and / or instructions that the processing device 110 may execute or use to execute the exemplary methods described in this specification. In some embodiments, the storage device 130 may include a mass storage device, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 130 may be implemented through a cloud platform.

[0032] In some embodiments, the storage device 130 may be connected to the network 140 to implement communication with one or more components (e.g., the processing device 110, the terminal device 120, etc.) in the region of interest generation system 100. One or more components in the region of interest generation system 100 may read data or instructions in the storage device 130 through the network 140. In some embodiments, the storage device 130 may be a part of the processing device 110, or may be independent and directly or indirectly connected to the processing device 120.

[0033] The network 140 may include any suitable network capable of facilitating information and / or data exchange of the region of interest generation system 100. In some embodiments, the network 140 may include one or more network access points. For example, the network 140 may include a wired and / or wireless network access point, such as a base station and / or an Internet exchange point, through which one or more components of the region of interest generation system 100 may connect to the network 140 to exchange data and / or information.

[0034] The medical device 150 may be a device for performing medical operations (such as medical image acquisition or treatment). In some embodiments, the medical device may include an imaging device and / or a treatment device. The imaging device 110 may scan an object located in a scanning area and generate a medical image related to the object. The object may include an organ, tissue, or lesion involved in a medical task. In some embodiments, the medical image generated by the imaging device 110 may be stored in the storage device 130 via the network 140. In some embodiments, the processing device 110 may obtain the medical image via the network 140. The treatment device may treat the object according to a medical plan specified by the user. In some embodiments, the medical plan may include a region of interest related to the object. In some embodiments, the medical device 150 may exchange data with other components (such as the processing device 110, the terminal device 120, and the storage device 130) in the region of interest generation system 100 via the network 140.

[0035] It should be noted that the above description of the region of interest generation system 100 is provided for illustrative purposes only and does not limit the scope of this specification. For those of ordinary skill in the art, various changes and modifications can be made based on the description of this specification. For example, the region of interest generation system 100 may include one or more additional components and / or one or more components of the region of interest generation system 100 described above may be omitted. For another example, two or more components of the region of interest generation system 100 may be integrated into a single component. The components of the region of interest generation system 100 may be implemented on two or more subcomponents.

[0036] Figure 21 is an exemplary module block diagram of a system for generating a region of interest according to some embodiments of the present specification. The system for generating a region of interest 100 may include an acquisition module 210 , a first determination module 220 , and a second determination module 230 .

[0037] The first acquisition module 210 may be configured to acquire a three-dimensional image of the object.

[0038] In some embodiments, the three-dimensional image can be obtained from a component in the region of interest generation system 100. For example, the acquisition module 210 can obtain a pre-stored three-dimensional image from the storage device 130. In some embodiments, the three-dimensional image can be obtained by calculating the two-dimensional medical image of the object through a preset algorithm. In some embodiments, the preset algorithm can be a reconstruction algorithm (for example, surface cover display, maximum density projection, surface reconstruction, etc.). In some embodiments, the preset algorithm may include but is not limited to a Shear-warp algorithm or a Marching Cubes (MC for short) algorithm. In some embodiments, the three-dimensional image may include an AR image or VR image displayed and interacted using an AR device or a VR device. In some embodiments, the object may refer to a part of the body of a subject for medical imaging, including but not limited to organs, tissues, or lesions. The three-dimensional image may include a three-dimensional contour of the object. In some embodiments, the three-dimensional contour of the object can be obtained by performing three-dimensional surface reconstruction on the two-dimensional medical image of the object based on the boundary contour line extraction algorithm of the Log differential operator.

[0039] The first determining module 220 may be configured to determine a two-dimensional contour of the object in a preset direction based on the three-dimensional image.

[0040] In some embodiments, the first determination module 220 may cut the three-dimensional image from a preset direction to obtain a two-dimensional contour. The preset direction may refer to the plane direction of the two-dimensional layer in space where the cutting operation is performed, and the two-dimensional layer is the cross section of the cutting operation. In some embodiments, the two-dimensional layer may include a coronal plane, a sagittal plane, a cross section (also called a cross section, a horizontal plane) or a cross section at any other angle in space. In some embodiments, the two-dimensional layer also has a specific plane position in space. In some embodiments, after cutting the three-dimensional image using the two-dimensional layer, a three-dimensional contour corresponding to the two-dimensional contour on the two-dimensional layer may be obtained.

[0041] The second acquisition module 230 may be configured to acquire a two-dimensional medical image of the object in the preset direction. In some embodiments, the two-dimensional medical image may be an image acquired by scanning the object using a medical imaging device, such as a CT image.

[0042] The two-dimensional image generation module 240 may be configured to generate a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour. In some embodiments, the two-dimensional image includes the image content of the two-dimensional medical image and the two-dimensional contour of the object. In some embodiments, the two-dimensional image generation module 240 may superimpose the two-dimensional contour located at the same plane position with the two-dimensional medical image to obtain a two-dimensional image.

[0043] The second determination module 250 may be configured to determine a region of interest from the two-dimensional image.

[0044] The region of interest may be a region related to the object selected by an operator in a two-dimensional image as a treatment reference. For example, in image-guided radiation therapy (IGRT), the region of interest may include a target region representing the lesion and an organ-at-risk region representing healthy tissue surrounding the lesion.

[0045] In some embodiments, the region of interest generation system 100 may further include a display module for displaying a three-dimensional image of an object and a two-dimensional image of a preset direction, etc. In some embodiments, the three-dimensional image may be displayed in different ways depending on the media medium. For example, when the media medium of the terminal device 120 is a plane display unit (e.g., a mobile phone, a display, etc.), the three-dimensional image may present a two-dimensional view of a corresponding perspective (e.g., a top view, a side view, a front view, etc.) on the terminal device 120 according to the corresponding operation of the operator. For another example, when the media medium of the terminal device 120 is a three-dimensional display unit (e.g., a virtual display device, a display enhancement device), the three-dimensional image may be presented to the operator in a corresponding manner (e.g., VR, AR, etc.). In some embodiments, the display module may also display the process of determining the region of interest.

[0046] Based on the system for generating a region of interest provided in this specification, a three-dimensional image can be converted into a two-dimensional image, and the operator only needs to perform operations in the two-dimensional direction to determine the region of interest, which can simplify the corresponding operations performed by the operator.

[0047] Figure 3 FIG. 1 is an exemplary flow chart of a method for generating a region of interest according to some embodiments of the present specification. Figure 3 As shown, process 300 includes the following steps.

[0048] Step 310 , obtaining a three-dimensional image of the object, wherein the three-dimensional image includes a three-dimensional outline of the object. In some embodiments, step 310 may be performed by a first obtaining module 310 .

[0049] In some embodiments, the three-dimensional image of the object may be a visual representation of a three-dimensional model of the object, and the three-dimensional image has a three-dimensional outline. In some embodiments, the way the three-dimensional model is acquired corresponds to the specific type of the object. For example, after taking a two-dimensional medical image of the object, the three-dimensional image of the object may be determined by medical image modeling techniques (e.g., surface masking display, maximum density projection, surface reconstruction, etc.). In some embodiments, the three-dimensional image may be directly stored in the storage device 130, and the processing device 110 may directly call the corresponding three-dimensional image from the storage device 130.

[0050] In some embodiments, a three-dimensional image of an object can be reconstructed and acquired based on a sequence of two-dimensional medical images of the object. The sequence of two-dimensional medical images can be acquired by continuous scanning of the object using a medical imaging device. For example, multiple CT images of the patient's lungs can be continuously taken based on scanning parameters, and the multiple CT images are a sequence of two-dimensional medical images. Specifically, multiple two-dimensional medical images of multiple slices of the object in a preset direction can be acquired, and then the three-dimensional image can be generated based on the two-dimensional medical images. Scanning parameters may include scanning direction, slice thickness, and slice spacing. Each slice may correspond to a two-dimensional medical image. The two-dimensional medical image corresponding to the slice includes an image representation of the slice of the object in the preset direction. As described herein, the two-dimensional medical image used to reconstruct the three-dimensional image may also be referred to as a first image.

[0051] The two-dimensional medical image may be one or more combinations of CT images, PET images, MRI images, etc. The preset direction (i.e., slice direction) of the two-dimensional medical image may refer to the scanning direction of the two-dimensional medical image. In some embodiments, the preset direction may be determined according to actual conditions. For example, if a patient has nodules in the lymph nodes, the two-dimensional medical image may be a plurality of CT images. Specifically, CT images may be continuously collected around the lymph nodes in a cross-sectional direction with a slice thickness of 1 cm to obtain a plurality of CT images containing complete lymph nodes. For another example, when taking lung slices, the CT image may be determined along the coronal plane direction of the object.

[0052] In some embodiments, the method of generating a three-dimensional image based on a two-dimensional medical image may include a surface cover display method, a maximum density projection method, a curved surface reconstruction method, etc. In some embodiments, the density field of the object can be determined based on the image content of the two-dimensional medical image, and reconstruction can be performed based on the density field to obtain a three-dimensional model of the object and represent it as a three-dimensional image. For example, the density field of the object and the three-dimensional model of the object can be determined based on the CT values ​​of various parts in the CT image. In some embodiments, the process of determining a three-dimensional image based on a two-dimensional medical image can be implemented by a machine learning algorithm.

[0053] In some embodiments, in order to improve the accuracy of the three-dimensional image, the three-dimensional image can be determined based on a variety of two-dimensional medical images. For example, CT images have a weaker ability to express soft tissues, and when determining the three-dimensional image, PET images and MRI images can be combined to improve the accuracy of the three-dimensional image.

[0054] In some embodiments, after acquiring the three-dimensional image, the terminal device 120 can present the three-dimensional image. For example, the three-dimensional image can be presented to the operator through VR / AR glasses 124 with 3D display function. After the operator wears the glasses, the three-dimensional image can be displayed in the operator's field of view in combination with the real scene through AR / VR.

[0055] Step 320 , determining a two-dimensional contour of the object in a preset direction based on the three-dimensional image. In some embodiments, step 320 may be performed by the first determining module 220 .

[0056] In some embodiments, the operator can select a preset direction through the terminal device. For example, the operator can select a preset direction from the coronal plane, the sagittal plane, and the transverse plane. For another example, the operator can also select the direction of a plane in space that is at any angle to a reference plane (e.g., the coronal plane, the sagittal plane, or the transverse plane) as the preset direction. For example, the operator can select the direction of a plane in space that is 30° to the transverse plane as the preset direction. In some embodiments, the preset direction selected by the operator can be the same as the preset direction of a two-dimensional medical image that generates a three-dimensional image.

[0057] In some embodiments, after the three-dimensional image is cut in a preset direction, a two-dimensional contour in the preset direction can be obtained. Cutting refers to the intersection of a plane and a solid (such as a three-dimensional image), and the plane obtained by cutting the solid by the plane is the cut surface. The two-dimensional contour can be the image contour on the cut surface generated by cutting the three-dimensional image using a two-dimensional layer in the preset direction.

[0058] In some embodiments, in order to fully describe the structure of the object, it is necessary to reflect the contour features of the object at different positions from multiple different cutting positions. Correspondingly, the two-dimensional layer of the preset direction may include multiple parallel layers with a certain thickness, wherein each layer may be spaced at a preset distance (e.g., 1 cm, 2 cm, etc.), which is equal to the slice thickness. When executing step 320, the preset direction may be determined, and the three-dimensional image may be cut in the preset direction at a preset spacing to obtain a two-dimensional contour. Among them, the value of the preset spacing may be specified by the operator or set by the system default. The preset spacing corresponding to adjacent truncated surfaces may be the same or different. In some embodiments, to facilitate subsequent operations, the preset spacing of the same object may be the same, that is, the three-dimensional image is sampled at equal intervals.

[0059] Figure 4Ais a schematic diagram of a three-dimensional image cutting process according to some embodiments of this specification, Figure 4B According to this manual Figure 4A In some embodiments, see Figure 4A During the cutting process, a three-dimensional image is provided with a plurality of cross sections (ie, two-dimensional layers in a preset direction), and the plurality of cross sections may correspond to the generation of a plurality of two-dimensional contours. Figure 4A The dashed line portion may represent a preset direction. For example, D7 may represent cutting a three-dimensional image on the D7 section to obtain a two-dimensional image on the cross section (ie, the preset direction).

[0060] In some embodiments, to ensure the accuracy of the operation, the two-dimensional contour of the object in a preset direction can be directly determined by the three-dimensional contour of the object. Specifically, the three-dimensional contour can be understood as the inner surface and / or outer surface of the object in the three-dimensional image, and the process of determining the two-dimensional contour from the three-dimensional contour can be to cut the three-dimensional contour at the cross-sectional position along the cross-sectional direction, and use the intersection line of the inner surface and / or outer surface and the cross-sectional area as the two-dimensional contour of the object in the two-dimensional image.

[0061] In some embodiments, in order to reduce the computational load, the two-dimensional contour of the object in the two-dimensional image can also be determined by the sampling points of the three-dimensional image. Figure 6 and its related description.

[0062] In some embodiments, the two-dimensional outlines of different objects may be presented in different ways. For example, to distinguish different objects, the color, line style, line thickness and other characterizing elements of the two-dimensional outlines of each object may be different.

[0063] Step 330 , obtaining a two-dimensional medical image of the object in the preset direction. Step 330 may be performed by the second obtaining module 230 .

[0064] In some embodiments, the second acquisition module 230 may acquire a two-dimensional medical image of the object from the medical device 150 or the storage device 130. In some embodiments, the two-dimensional medical image acquired by the second acquisition module 230 may be a two-dimensional medical image used to generate a three-dimensional image.

[0065] In some embodiments, the preset direction of the two-dimensional medical image acquired by the second acquisition module 230 is the same as the preset direction of the section of the three-dimensional image. In some embodiments, the section position of the two-dimensional layer when the three-dimensional image is cut is the same as the position of the slice when the medical image of the object is taken. In this case, the two-dimensional contour of the object in the preset direction can correspond to the two-dimensional medical image one by one.

[0066] Step 340 , generating a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour. Step 340 may be performed by the two-dimensional image generating module 240 .

[0067] In some embodiments, the two-dimensional image includes the image content of the two-dimensional medical image and the two-dimensional contour of the object. In some embodiments, the two-dimensional layer (i.e., the cross section) corresponding to the two-dimensional contour of the object can correspond to the two-dimensional medical image one-to-one, that is, the two-dimensional layer coincides with the two-dimensional medical image, and the two-dimensional image generation module 240 can superimpose the two-dimensional medical image of the corresponding slice position on the two-dimensional contour of the truncated three-dimensional image to generate the two-dimensional image corresponding to the slice, and the two-dimensional image can include the contour of the three-dimensional image of the object on the cross section and the corresponding two-dimensional medical image. In some embodiments, the one-to-one correspondence between the two-dimensional layer and the two-dimensional medical image may include a correspondence between the same plane position in a preset direction. In some embodiments, the one-to-one correspondence between the two-dimensional layer and the two-dimensional medical image may also include a correspondence between the same slice thickness in a preset direction. As described herein, the two-dimensional image generated based on the two-dimensional medical image and the two-dimensional contour may also be referred to as a second image.

[0068] It should be noted that the two-dimensional image (i.e., the second image) generated based on the two-dimensional medical image and the two-dimensional contour is a different image from the two-dimensional medical image (i.e., the first image) used to generate the three-dimensional image. In some embodiments, the preset direction corresponding to the second image may be the same as the preset direction corresponding to the first image. In some embodiments, the slice thickness corresponding to the second image may be the same as or different from the slice thickness corresponding to the first image.

[0069] In some embodiments, according to the needs of the actual medical plan, the two-dimensional layer may not correspond to the two-dimensional medical image one-to-one, that is, the two-dimensional layer and the two-dimensional medical image do not overlap. To facilitate the generation of a two-dimensional image, some embodiments of the present specification also provide a method for fitting a two-dimensional medical image so that each two-dimensional layer corresponds to a two-dimensional medical image. In some embodiments, the method for fitting a two-dimensional medical image may have the following steps: determining the position information of the two-dimensional layer of a preset direction corresponding to the two-dimensional contour in the three-dimensional image; based on the position information, determining the two-dimensional medical fitting image corresponding to the two-dimensional layer of the preset direction by a preset interpolation algorithm; generating a two-dimensional image based on the two-dimensional medical fitting image and the two-dimensional contour.

[0070] In some embodiments, the position information may include the slice position of the two-dimensional layer. In some embodiments, the slice position of the two-dimensional layer may include the slice thickness. In some embodiments, the position information of the two-dimensional layer of the cut three-dimensional image may be different from the position information of the two-dimensional medical image. For example, if the slice thickness of the two-dimensional layer is 1.5 cm and the slice thickness of the two-dimensional medical image is 1 cm, then each two-dimensional layer cannot correspond to a two-dimensional medical image that completely overlaps with it.

[0071] In some embodiments, a two-dimensional medical fitting image corresponding to the position information of the two-dimensional layer can be obtained by calculating the two-dimensional medical image, and the two-dimensional medical fitting image contains medical information in one or more two-dimensional medical images. In some embodiments, the processing device can fit the two-dimensional medical image according to a preset interpolation algorithm to obtain a two-dimensional medical fitting image, and superimpose the two-dimensional contour with the fitted two-dimensional medical fitting image of the corresponding slice position to obtain each two-dimensional image. For example, the slice thickness of the two-dimensional layer is 1.5 cm, and the slice thickness of the two-dimensional medical image is 1 cm. The fitting result of the two-dimensional medical image when the slice thickness is 1.5 cm can be calculated by the interpolation algorithm, and the fitted two-dimensional medical fitting image is used as the corresponding two-dimensional medical image of the two-dimensional layer. Superimpose the two-dimensional contour to generate a two-dimensional image. In some embodiments, the preset interpolation algorithm may include one of polynomial interpolation (for example, general polynomial interpolation, Lagrange interpolation, Newton interpolation, etc.), piecewise interpolation (for example, Hermite interpolation, piecewise cubic Hermite interpolation, cubic spline interpolation, etc.). In some embodiments, the preset interpolation algorithm can be implemented by machine learning.

[0072] Step 350 , determining the region of interest from the two-dimensional image. In some embodiments, step 350 may be performed by the second determination module 250 .

[0073] The region of interest may be a region selected by a user that is relevant to the medical plan of the object. In some embodiments, the user may be an operator of the system 100. For example, the operator may select the region where the object (such as a lesion) is located and the region of an organ at risk as the region of interest, and mark the region of interest to distinguish it from other irrelevant parts in the two-dimensional image.

[0074] In some embodiments, the two-dimensional contour of the object can be determined as the region of interest. In some embodiments, due to the accuracy of the imaging device and the actual situation, the two-dimensional contours of each object in the two-dimensional medical image may not be clear or accurate enough. For example, one of the pathological characteristics of a 4b type thyroid nodule in CT is that the nodule contour is not clear. For another example, Figure 5The contrast angle between the middle and lower vena cava and the surrounding tissues makes the contours fuzzy. Based on this situation, the 2D contour of the object may not accurately correspond to the actual contour of the object, which requires the operator to adjust the 2D contour on the 2D image according to the actual situation to determine the area of ​​interest.

[0075] In some embodiments, the operator can modify the two-dimensional contour based on actual conditions (e.g., clinical manifestations) to determine the region of interest. For example, when the target area is severely deteriorating, the contour of the target area can be enlarged so that the radiation dose to the target area can be increased in subsequent medical treatment. In the specific execution process, the user instructions can be obtained first; then, based on the user instructions, the two-dimensional contour in the two-dimensional image is edited to determine the region of interest. User instructions may include object selection instructions, contour modification instructions, etc. For example, user instructions may include relevant instructions to hide unselected objects and adjust the two-dimensional contour of selected objects. The processing device 110 may adjust the two-dimensional contour based on the user instructions to determine the region of interest. For more information about user instructions, see Figure 5 Related description.

[0076] In some embodiments, the two-dimensional image (e.g., the two-dimensional image with or without a determined region of interest) may be presented by the terminal device 120. In some embodiments, the two-dimensional image may be directly presented in the terminal device 120, and its specific presentation interface may refer to the present application. Figure 5 The presentation method shown.

[0077] In some embodiments, there may be multiple objects, each of which may correspond to multiple two-dimensional images, and the terminal device 120 may present a two-dimensional image based on each of the multiple objects. For example, the multiple objects may include lungs and stomachs, and the multiple two-dimensional images corresponding to the lungs are different from the multiple two-dimensional images corresponding to the stomach. The operator may display the two-dimensional image corresponding to the lungs or the stomach separately in the terminal device 120.

[0078] In some embodiments, the operator can set a preset presentation form of the object. For example, the operator can set it to present only the two-dimensional outline of the object. In this case, the two-dimensional outline of each object in the cross section of the three-dimensional image is presented on the two-dimensional image.

[0079] Combination Figure 4B , each object in the two-dimensional image can be presented in the form of a two-dimensional outline. Figure 4B It is a cross section at the D7 section position, and the objects specifically included in the cross section are: trunk 400 , right liver lobe 410 , left liver lobe 420 , stomach 430 , spleen 440 , main trunk of portal vein 450 , tumor 460 , inferior vena cava 470 , and abdominal aorta 480 .

[0080] In some embodiments, the terminal device 120 may also present the process of editing the two-dimensional contour to determine the region of interest, so that the user can observe and adjust in real time during the editing process. In some embodiments, the terminal device 120 may also present the generated region of interest. In some embodiments, the terminal device 120 may also present the generated region of interest in a three-dimensional image, so that the operator can observe the accuracy of the region of interest in combination with the three-dimensional image.

[0081] In some embodiments, in order to further reduce the workload of operators and improve efficiency, the second determination module 230 can also automatically edit the two-dimensional contour and determine the region of interest. Figure 7 Related description.

[0082] Figure 5 is a schematic diagram of a user interaction interface according to some embodiments of this specification. Figure 5 As shown, the user interaction interface may include an operation part and a display part. The operation part may provide a list of regions of interest and provide the user with a function of editing the list of regions of interest, and the display part may perform corresponding display based on the user's operation on the list of regions of interest.

[0083] In some embodiments, when there are multiple objects, the operator can determine the region of interest in the two-dimensional image based on each of the multiple objects, and the region of interest corresponding to each object can be presented in multiple different two-dimensional images. In some embodiments, the user interaction interface can include a list of regions of interest (i.e., Figure 5 The ROI list shown in the figure) may include object names corresponding to the respective ROIs, and the user may select an object name in the list to enable the user interaction interface to present the corresponding two-dimensional image and the ROI in the two-dimensional image.

[0084] In some embodiments, the list of regions of interest (i.e., ROI list) may include an operation control for providing an operator with buttons that can implement interactive operations. For example, the operation control may include a selection button, a visible button, an edit button, an add button, a delete button, a lock button, etc. In some embodiments, the selection button may be used to select one or more of the regions of interest from a plurality of regions of interest. In some embodiments, the visible button may be used to switch the visible state of the region of interest so as to display a two-dimensional image including the region of interest in the display portion. In some embodiments, the edit button is used to edit the region of interest. In some embodiments, the add button may be used to add a new region of interest to the region of interest list. In some embodiments, the delete button may be used to delete the region of interest from the region of interest list. In some embodiments, the lock button is used to change the editable state of the region of interest.

[0085] Specifically, each object can be added to the region of interest list through an add button, and the operator can determine whether each object is displayed in the two-dimensional image through the list, for example, through a visible button in the list to perform a display operation.

[0086] In some embodiments, the operator can further edit the shape of each region of interest outline through the edit button to redefine the region of interest. For example, when the operator clicks the edit button, the outline of the corresponding region of interest can be changed to a Bezier curve, whereby the operator can change the outline of the region of interest by changing the number, position, and arc of the nodes of the Bezier curve.

[0087] In some embodiments, in order to prevent the contour of the region of interest from being affected by erroneous operations, the contour of the region of interest may be locked using a lock button. In the locked state, no editing operations can be performed on the region of interest.

[0088] In some embodiments, the two-dimensional image may further include auxiliary information, such as relevant information such as date, patient number, device code, etc. The user interaction interface may display the auxiliary information in the two-dimensional image.

[0089] In some embodiments, considering that a large number of two-dimensional images are presented, there may be multiple (for example, dozens) of two-dimensional images between regions of interest at different locations. To facilitate the operator to quickly locate, the operator can directly select the region of interest from the aforementioned region of interest list. The user interaction interface can respond to the user operation and jump to the two-dimensional image in the corresponding range of the region of interest selected by the user. Within the corresponding range, the user can perform traversal operations, for example, can use gestures or slide the mouse wheel to flip the two-dimensional image up and down.

[0090] Figure 6 is an exemplary flow chart of determining a two-dimensional contour of an object according to some embodiments of this specification. Figure 6 , process 600 includes the following steps.

[0091] Step 610: Generate a preset number of sampling points based on the three-dimensional contour of the three-dimensional image.

[0092] The sampling points may be points on the three-dimensional contour of the object in the three-dimensional image, and are used to characterize the three-dimensional contour of the object in the three-dimensional image by sampling, so as to reduce the amount of calculation and thus reduce the load of the processing device 110 .

[0093] In some embodiments, the sampling points in step 610 can be determined by a sampling algorithm, and the sampling algorithm can include one or more of a uniform distribution sampling algorithm, a discrete distribution sampling algorithm, a Box-Muller algorithm, a rejection sampling algorithm, a MCMC sampling algorithm, and a Gibbs sampling algorithm.

[0094] In some embodiments, the sampling points may be determined geometrically. For example, the three-dimensional image may be divided into a plurality of grids, and the three-dimensional image may be divided into a plurality of regions according to the grids, and each region may be used as a sampling point. Figure 4A In the human body model shown, the intersections of the horizontal and vertical lines may be sampling points.

[0095] Step 620: Based on the preset number of sampling points, determine a plurality of sampling points corresponding to the two-dimensional plane of the preset direction.

[0096] In some embodiments, when the sampling point coincides with a two-dimensional plane in a preset direction, the sampling point on the two-dimensional plane in the preset direction may be directly used as the sampling point of the two-dimensional plane. In some embodiments, when the sampling point does not coincide with a two-dimensional plane in a preset direction, the sampling points of the two-dimensional plane in the preset direction within a preset projection range perpendicular to the two-dimensional plane direction may be projected onto the two-dimensional plane, wherein the preset projection range may be any value not greater than half of the truncation interval (the distance between the two-dimensional planes in each preset direction).

[0097] In some embodiments, to avoid sampling points not coinciding with the two-dimensional plane of the preset direction, sampling may be performed according to the two-dimensional plane of the preset direction, that is, the three-dimensional contour is sampled only at the position corresponding to the two-dimensional plane of the preset direction.

[0098] Step 630: Generate the two-dimensional contour based on the multiple sampling points.

[0099] The sampling points of each two-dimensional layer may be a set of points constituting the two-dimensional layer, and the sampling points may be converted into a two-dimensional contour based on a point set to line algorithm. The two-dimensional contours of each object in the three-dimensional image are calculated separately without affecting each other. The point set to line algorithm may be an algorithm that connects points into a smooth curve based on the positional relationship of the points in a plane. In some embodiments, the point set to line algorithm may be executed by software (such as ArcGIS).

[0100] In some embodiments, in order to further reduce the workload of operators and improve efficiency, some embodiments of this specification also provide a method process 700 for automatically editing a two-dimensional contour and determining a region of interest. Figure 7 is an exemplary flow chart of determining an area of ​​interest according to some embodiments of this specification, see Figure 7 , process 700 may include the following steps.

[0101] Step 710: Obtain relevant information of the user, wherein the relevant information of the user may include the identity information of the operator (eg, doctor ID).

[0102] In some embodiments, the user's relevant information may be stored in the storage device 130 , and the processing device 110 may call the relevant information accordingly to implement step 710 .

[0103] Step 720: Acquire historical operation information of the user based on the relevant information.

[0104] In some embodiments, the user's historical operation information can be stored in the storage device 130, and the processing device 110 can call the historical operation information according to the user's relevant information (e.g., doctor ID) to implement step 720. The historical operation information can be relevant information when the operator determines the region of interest in the historical operation. In some embodiments, the historical operation information can include historical objects, historical two-dimensional contours, historical editing records, and historical regions of interest. The historical object can be the object corresponding to the operator when determining the region of interest in the historical operation. In some embodiments, the historical object can come from different patients. The historical two-dimensional contour can be the two-dimensional contour of the historical object in the historical operation. The historical editing record can be the operator's editing log of the two-dimensional contours of each historical object. The historical region of interest can be the region of interest determined by the operator in the historical operation.

[0105] Step 730: Determine a region of interest based on the historical operation information.

[0106] In some embodiments, step 730 may determine the region of interest by matching the current information with the historical operation information. The method of matching the current information with the historical operation information may include the following steps:

[0107] First, based on the object matching corresponding historical objects in the historical operation information, a set of one or more historical operation information is obtained. In some embodiments, each historical operation information corresponds to a historical object.

[0108] In some embodiments, the object and the historical object can be input into the first feature recognition network to obtain the feature vectors of the object and the historical object, and then the historical object matching the object is determined based on the feature vector, thereby determining the historical operation information corresponding to the historical object, wherein the first feature recognition network can be a trained convolutional neural network. The shape, type and at least one of the relevant patient information of the object can be used as the input of the first feature recognition network, and based on the input, the first feature recognition network can output a set of one or more historical objects and / or operation information. In some embodiments, the first feature recognition network can be a neural network constructed based on a machine learning algorithm, such as a CNN or RNN network. In some embodiments, the first feature recognition network can also be a matching algorithm, which can match the corresponding parts of the historical objects and / or historical operation information based on text, semantics and other information.

[0109] Then, historical two-dimensional contours with similar contours are matched in multiple historical target parts based on the two-dimensional contours, so as to determine the reference historical editing record.

[0110] In some embodiments, the two-dimensional contour and the historical two-dimensional contour can be input into a second feature recognition network to obtain feature vectors of the two-dimensional contour and the historical two-dimensional contour, and then the historical operation information matching the two-dimensional contour is determined based on the feature vector as a reference historical editing record. In some embodiments, the shape of the two-dimensional contour can be characterized as a feature vector, and the corresponding similar shapes can be characterized as similar feature vectors. For example, two contours whose feature vectors differ by less than 5% can be regarded as contours with similar shapes.

[0111] Next, current editing data is determined based on the historical editing data in the reference historical editing record. The current editing data can be used as an editing instruction for automatically editing the two-dimensional contour to determine the region of interest. In some embodiments, the current editing data may include editing points (such as nodes of a Bezier curve) for editing the two-dimensional contour and the editing amount of each editing point.

[0112] In some embodiments, the historical editing data may be used as the current editing data.

[0113] In some embodiments, considering that there may be certain differences between the two-dimensional contour and the historical two-dimensional contour, the historical edit data may be adjusted based on the difference to generate the current edit data. The difference between the two-dimensional contour and the historical two-dimensional contour may be a difference in position, a difference in shape, etc., and a mapping may be generated based on the difference to adjust the historical edit data. For example, if the two-dimensional contour and the historical two-dimensional contour differ in position by 5 cm, and the image sizes of the two contours differ by 10%, the historical edit data may be displaced by 5 cm and scaled by 10% according to the actual situation.

[0114] Finally, the region of interest is determined based on the current editing data.

[0115] In some embodiments, a user instruction may be generated based on the current editing data to cause the processing device 110 to respond by determining a region of interest.

[0116] In some embodiments, based on the regions of interest determined in the embodiments of this specification, doctors can determine the radiation dose of each region of interest and estimate the received dose of each region of interest so that the lesion area meets the treatment requirements while avoiding the organ-at-risk area from exceeding the tolerance dose.

[0117] The beneficial effects that may be brought about by the embodiments of the present application include but are not limited to: (1) displaying a three-dimensional image as a two-dimensional image, making it easier for operators to perform related operations; (2) automatically generating a region of interest, reducing the workload of related operators. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0118] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of the present application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements and amendments to the present application. Such modifications, improvements and amendments are suggested in the present application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of the present application.

[0119] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

Claims

1. A method for generating a region of interest, It is characterized in that The method comprises: Acquire a three-dimensional image of an object, the three-dimensional image including a three-dimensional contour of the object; Determining a two-dimensional contour of the object in a preset direction based on the three-dimensional image; Acquiring a two-dimensional medical image of the object in the preset direction; generating a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour; and A region of interest is determined from the two-dimensional image.

2. The method according to claim 1, It is characterized in that Determining the two-dimensional contour of the object in a preset direction based on the three-dimensional image includes: determining the preset direction; and The three-dimensional image is cut using the two-dimensional layer in the preset direction at a preset interval to obtain the two-dimensional contour.

3. The method according to claim 2, It is characterized in that The three-dimensional image is generated by calculating a plurality of two-dimensional medical images in the preset direction using a preset algorithm.

4. The method according to claim 2, It is characterized in that The two-dimensional layer overlaps with the slice position of the two-dimensional medical image.

5. The method according to claim 2, It is characterized in that The two-dimensional layer does not overlap with the slice position of the two-dimensional medical image; and generating a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour includes: Determine position information of the two-dimensional layer of the preset direction corresponding to the two-dimensional contour in the three-dimensional image; Based on the position information, determining a two-dimensional medical fitting image corresponding to the two-dimensional layer in the preset direction by a preset interpolation algorithm; A two-dimensional image is generated based on the two-dimensional medical fitting image and the two-dimensional contour.

6. The method according to claim 4 or 5, It is characterized in that The generating of the two-dimensional image based on the two-dimensional medical image and the two-dimensional contour comprises: superimposing the two-dimensional medical image or the two-dimensional medical fitting image corresponding to the slice position on the two-dimensional contour to generate the two-dimensional image.

7. The method according to claim 2, It is characterized in that The determining the two-dimensional contour of the object in the preset direction includes: generating a preset number of sampling points based on the three-dimensional contour of the three-dimensional image; Based on the preset number of sampling points, determining a plurality of sampling points corresponding to the two-dimensional plane of the preset direction; The two-dimensional profile is generated based on the plurality of sampling points.

8. The method according to claim 1, It is characterized in that Determining the region of interest based on the two-dimensional image comprises: Obtaining user instructions; and The two-dimensional contour in the two-dimensional image is edited based on the user instruction to determine the region of interest.

9. The method according to claim 1, It is characterized in that The three-dimensional image includes an AR or VR image.

10. A system for generating a region of interest, It is characterized in that The system comprises: A first acquisition module, configured to acquire a three-dimensional image of an object, wherein the three-dimensional image includes a three-dimensional contour of the object; A first determining module, configured to determine a two-dimensional contour of the object in a preset direction based on the three-dimensional image; A second acquisition module, used for acquiring a two-dimensional medical image of the object in the preset direction; a two-dimensional image generating module, configured to generate a two-dimensional image based on the two-dimensional medical image and the two-dimensional contour; and The second determining module is used to determine a region of interest from the two-dimensional image.

Images