Image processing method and system, neural network model training method and medical imaging system

Abstract

The invention discloses an image processing method and system, a neural network model training method and a medical imaging system. A method for image processing includes: acquiring original projection data of an examined object, the original projection data being obtained by scanning the examined object by a medical imaging system; constructing an input feature, wherein the input feature comprises trend information of the original projection data; and using the neural network model to generate enhanced projection data of the examination object based on the input features, wherein the enhanced projection data has a higher resolution than the original projection data and is used to reconstruct a medical image of the examination object.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image processing method and system, a neural network model training method, and a medical imaging system. Background Art

[0002] Imaging technology allows for the non-invasive acquisition of images of internal structures or features of a subject, such as a patient. Digital X-ray imaging systems generate digital data that can be reconstructed into radiographic images, such as in computed tomosynthesis (CT) or digital breast tomosynthesis (DBT) imaging processes. In a digital X-ray imaging system, radiation from a source is directed toward the subject. A portion of the radiation passes through the subject and strikes a detector. The detector includes an array of discrete picture elements or detector pixels, and is processed based on the amount or intensity of radiation striking each pixel area to obtain projection data. The complete projection data can reconstruct accurate slice images for diagnosis. These images are used to identify and / or examine internal structures and organs in a patient's body. If the image resolution is higher, the internal structures and organs can be more clearly distinguished, resulting in a more accurate diagnosis. Summary of the invention

[0003] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

[0004] According to a first aspect of the present disclosure, a method for image processing is provided, comprising: acquiring original projection data of an examination object, wherein the original projection data is obtained by scanning the examination object by a medical imaging system; constructing input features, wherein the input features include trend information of the original projection data; and using a neural network model to generate enhanced projection data of the examination object based on the input features, wherein the enhanced projection data has a higher resolution than the original projection data and is used to reconstruct a medical image of the examination object.

[0005] In one embodiment, the raw projection data is three-dimensional projection data acquired by a detector of the medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector when the inspection object moves toward or out of the medical imaging system, the channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object, and the viewing angle direction indicates the angle at which the detector acquires the raw projection data at different positions around the inspection object.

[0006] In one embodiment, the trend information of the original projection data includes one or more of the following items: projection data trend information on at least one dimension; and frequency trend information in a specific order obtained by filtering the original projection data of at least one position in the at least one dimension using at least two kernel functions with different frequencies.

[0007] In one embodiment, the trend information of the original projection data includes the projection data trend information presented by the projection data block at at least one position in the at least one dimension and the frequency trend information presented by the filtered projection data block obtained by filtering the projection data block through at least two kernel functions of different frequencies.

[0008] In one embodiment, the projection data block includes raw projection data in a plane formed by other two dimensions at a position of one dimension of the raw projection data.

[0009] In one embodiment, constructing the input feature includes: constructing the trend information of the original projection data as an input channel of the input feature.

[0010] In one embodiment, the medical imaging system is a computed tomography medical imaging system, a positron emission tomography medical imaging system, or a positron emission tomography medical imaging system.

[0011] According to a second aspect of the present disclosure, a method for training a neural network model is provided, comprising: acquiring a training data set, the training data set comprising training original projection data and training enhanced projection data, wherein the training original projection data and the training enhanced projection data can each be used to reconstruct a medical image of an examination object, the training enhanced projection data has a higher resolution than the training original projection data, and the training enhanced projection data is used as a true value of an output of the neural network model; constructing training input features, the training input features comprising trend information of the training original projection data; using the neural network model to generate a prediction result of enhanced projection data having a higher resolution than the training original projection data based on the training input features; calculating a loss function between the prediction result and the true value; and updating the parameters of the neural network model based on the loss function to obtain a trained neural network model.

[0012] In one embodiment, the training raw projection data is three-dimensional projection data acquired by the detector of the medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector when the examination object moves toward or out of the medical imaging system, the channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the examination object, and the viewing angle direction indicates the angle at which the detector acquires the training raw projection data at different positions around the examination object.

[0013] In one embodiment, the trend information of the training original projection data includes one or more of the following items: projection data trend information on at least one dimension; and frequency trend information in a specific order obtained by filtering the training original projection data of at least one position of the at least one dimension using at least two kernel functions with different frequencies.

[0014] In one embodiment, constructing the training input feature includes: constructing the trend information of the training original projection data as an input channel of the training input feature.

[0015] In one embodiment, the loss function includes at least one of the following: a mean absolute error loss function, an average structural similarity index measurement loss function, and a perceptual loss function.

[0016] According to a third aspect of the present disclosure, a system for image processing is provided, comprising a neural network model, wherein: the neural network model receives trend information of original projection data acquired by a medical imaging system scanning an object for examination as input features and outputs enhanced projection data, wherein the enhanced projection data has a higher resolution than the original projection data; the neural network model comprises: a shallow feature extraction layer, for performing feature extraction on the input features using a convolution layer to obtain shallow features; a deep feature extraction layer, for performing feature extraction on the shallow features using at least one cascaded residual group, convolution layer and summation module to obtain deep features; and an upsampling layer, for upsampling the deep features into the enhanced projection data.

[0017] In one embodiment, each residual group of the deep feature extraction layer includes: a plurality of cascaded residual blocks, each residual block being used to extract deep features of different levels from the input of the residual group; a splicing layer being used to splice the deep features extracted by each residual block to obtain spliced ​​deep features; and a convolution layer being used to perform a convolution operation on the spliced ​​deep features to obtain the output of the residual group.

[0018] In one embodiment, the residual block includes: a plurality of parallel convolutional layers of different sizes, each convolutional layer being used to perform a convolution operation on the input of the residual block to obtain a convolution result; a plurality of activation function modules, each activation function module being cascaded with one of the plurality of parallel convolutional layers of different sizes, and being used to apply an activation function to the convolution result of the corresponding convolutional layer to obtain a local feature; a splicing layer being used to splice the local features of each activation function module to obtain a spliced ​​local feature; a final convolutional layer being used to perform a convolution operation on the spliced ​​local feature to obtain a final convolution result; and a summing module being used to add the final convolution result to the input of the residual block to obtain an output of the residual block.

[0019] In one embodiment, the raw projection data is three-dimensional projection data acquired by a detector of the medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector when the inspection object moves toward or out of the medical imaging system, the channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object, and the viewing angle direction indicates the angle at which the detector acquires the raw projection data at different positions around the inspection object.

[0020] In one embodiment, the trend information of the original projection data includes one or more of the following items: projection data trend information on at least one dimension; and frequency trend information in a specific order obtained by filtering the original projection data of at least one position in the at least one dimension using at least two kernel functions with different frequencies.

[0021] According to a fourth aspect of the present disclosure, a medical imaging system is provided, comprising: a scanning device configured to acquire raw projection data of an inspection object; and a processor configured to execute any of the aforementioned methods.

[0022] According to a fifth aspect of the present disclosure, a non-transitory computer-readable medium is provided, on which instructions are stored, and the instructions can be executed by a processor to implement any of the methods described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention may be better understood by describing exemplary embodiments of the present invention in conjunction with the accompanying drawings, in which:

[0024] Figure 1 shows a schematic diagram of an exemplary CT system configured for CT imaging;

[0025] Figure 2 Shows something like Figure 1 An exemplary imaging system of a CT system;

[0026] Figure 3 A schematic diagram of a CT system when examining a patient is shown;

[0027] Figure 4 A flowchart of an image processing method according to an embodiment of the present disclosure is shown;

[0028] Figure 5 A schematic diagram showing raw projection data according to an embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram showing a trend information of projection data in a viewing angle direction of original projection data according to an embodiment of the present disclosure is shown;

[0030] Figure 7 A schematic diagram showing frequency trend information of raw projection data according to an embodiment of the present disclosure;

[0031] Figure 8 A method for performing batch processing in a channel-row plane according to an embodiment of the present disclosure is shown;

[0032] Fig. 9 A flowchart of a method for training a neural network model according to an embodiment of the present disclosure is shown;

[0033] Fig.10 A flowchart of a method for image processing according to another embodiment of the present disclosure is shown;

[0034] FIG. 11A to FIG. 11C A schematic diagram of a neural network model according to an embodiment of the present disclosure is shown;

[0035] FIG. 12A to FIG. 12D A comparison diagram showing a reconstructed image generated based on original projection data and a reconstructed image generated based on enhanced projection data; and

[0036] Fig.13 An example block diagram of a computing device according to one embodiment of the present disclosure is shown.

[0037] In the accompanying drawings, similar components and / or features may have the same numerical reference number. In addition, components of the same type may be distinguished by following the reference number with a letter that distinguishes between similar components and / or features. If only the first numerical reference number is used in the specification, the description applies to any similar component and / or feature having the same first numerical reference number, regardless of the letter subscript. DETAILED DESCRIPTION

[0038] The specific embodiments of the present invention will be described below. It should be noted that in the specific description of these embodiments, in order to provide a concise description, it is impossible for this specification to provide a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one embodiment to another. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by the present invention, some changes such as design, manufacturing or production based on the technical content disclosed in this disclosure are just conventional technical means, and should not be understood as insufficient content of this disclosure.

[0039] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0040] For purposes of this disclosure, the phrase "A and / or B" means (A), (B), or (A and B). For purposes of this disclosure, the phrase "A, B, and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).

[0041] Unless otherwise defined, the technical or scientific terms used in the claims and the specification shall have the common meanings understood by persons with ordinary skills in the technical field to which the invention belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before "include" or "comprising" include the elements or objects listed after "include" or "comprising" and their equivalent elements, and do not exclude other elements or objects.

[0042] The following will take the example of Figures 1 to 13To describe the embodiments of the present disclosure. Although the CT system is described by way of example, it should be understood that the technology of the present disclosure is widely applicable to various non-destructive detection fields. The technology of the present disclosure may also be useful when applied to images collected using other imaging modalities, such as X-ray imaging systems, magnetic resonance imaging (MRI) systems, positron emission tomography (PET) imaging systems, single photon emission computed tomography (SPECT) imaging systems, and combinations thereof (e.g., multimodal imaging systems, such as PET / CT, PET / MR or SPECT / CT imaging systems). Exemplarily, the following describes the embodiments of the present disclosure in conjunction with X-ray computed tomography (CT) imaging. Those skilled in the art will appreciate that the embodiments of the present disclosure may also be applicable to other medical imaging.

[0043] Figure 1 A schematic diagram of an exemplary CT system 100 configured for CT imaging is shown. Specifically, the CT system 100 is configured to image a subject 112 (such as a patient, an inanimate object, one or more manufactured parts) and / or a foreign object (such as a dental implant, a stent, and / or a contrast agent present in the body). In one embodiment, the CT system 100 includes a gantry 102, which in turn may also include at least one X-ray source 104, which is configured to project an X-ray radiation beam 106 (see Figure 2 ) for imaging a subject 112 lying on an examination table 114. Specifically, the X-ray source 104 is configured to project an X-ray radiation beam 106 toward a detector array 108 positioned on an opposite side of the gantry 102. Figure 1 A single X-ray source 104 is depicted, but in certain embodiments, multiple X-ray sources and detectors may be employed to project multiple X-ray radiation beams for acquiring projection data at different energy levels corresponding to the patient. In some embodiments, the X-ray source 104 may implement dual-energy gem spectrum imaging (GSI) by rapid peak kilovolt (kVp) switching. In some embodiments, the X-ray detector employed is a photon counting detector capable of distinguishing X-ray photons of different energies. In other embodiments, two sets of X-ray sources and detectors are used to generate dual-energy projections, wherein one set of X-ray sources and detectors is set to low kVp and the other set to high kVp. It should therefore be understood that the methods described herein may be implemented with single energy acquisition techniques as well as dual energy acquisition techniques.

[0044] In certain embodiments, the CT system 100 further includes an image processing unit 110 configured to reconstruct an image of a target volume of the subject 112 using an iterative or analytical image reconstruction method. For example, the image processing unit 110 may use an analytical image reconstruction method such as filtered back projection (FBP) to reconstruct an image of the target volume of the patient. As another example, the image processing unit 110 may use an iterative image reconstruction method (such as advanced statistical iterative reconstruction (ASIR), conjugate gradient (CG), maximum likelihood expectation maximization (MLEM), model-based iterative reconstruction (MBIR), etc.) to reconstruct an image of the target volume of the subject 112. As further described herein, in some examples, in addition to the iterative image reconstruction method, the image processing unit 110 may also use an analytical image reconstruction method (such as FBP).

[0045] In some CT imaging system configurations, an X-ray source projects a cone-shaped X-ray radiation beam that is collimated to lie in the XYZ plane of a Cartesian coordinate system and is often referred to as the "imaging plane". The X-ray radiation beam passes through an object being imaged, such as a patient or subject. The X-ray radiation beam, after being attenuated by the object, impinges upon an array of detector elements. The intensity of the attenuated X-ray radiation beam received at the detector array depends on the attenuation of the X-ray radiation beam by the object. Each detector element of the array produces a separate electrical signal that is a measurement of the X-ray beam attenuation at the detector location. The attenuation measurements from all detector elements are collected individually to produce a transmission profile.

[0046] In some CT systems, a gantry is used to rotate the X-ray source and the detector array around the object to be imaged in the imaging plane so that the angle at which the X-ray beam intersects the object is constantly changing. A set of X-ray radiation attenuation measurements (e.g., projection data) from the detector array at one gantry angle is called a "view". A "scan" of an object includes a set of views made at different gantry angles or viewing angles during one rotation of the X-ray source and the detector. It is conceivable that the benefits of the methods described herein are derived from medical imaging modalities other than CT, so as used herein, the term "view" is not limited to the use described above with respect to projection data from one gantry angle. The term "view" is used to mean a data acquisition whenever there are multiple data acquisitions from different angles (whether from CT, positron emission tomography (PET) or single photon emission CT (SPECT) acquisitions), and / or any other modality (including modalities yet to be developed) and their combination in fusion embodiments.

[0047] The projection data is processed to reconstruct an image corresponding to a two-dimensional slice acquired through the object, or in some examples where the projection data includes multiple views or scans, an image corresponding to a three-dimensional image of the object. A method for reconstructing an image from a set of projection data is known in the art as a filtered back projection technique. Transmission and emission tomography reconstruction techniques also include statistical iterative methods such as maximum likelihood expectation maximization (MLEM) and ordered subset expectation reconstruction techniques and iterative reconstruction techniques. The method converts attenuation measurements from the scan into an integer called a "CT number" or "Hou's unit", which is used to control the brightness of the corresponding pixel on the display device.

[0048] To reduce the total scan time, a "helical" scan can be performed. To perform a "helical" scan, the patient is moved while data for a specified number of slices is acquired. Such systems produce a single spiral from a cone beam helical scan. The spiral mapped out by the cone beam produces projection data from which an image in each specified slice can be reconstructed.

[0049] As used herein, the phrase "reconstructing an image" is not intended to exclude embodiments in which data representing an image is generated without generating a visible image. Thus, as used herein, the term "image" broadly refers to both a visible image and data representing a visible image. However, many embodiments generate (or are configured to generate) at least one visible image.

[0050] Figure 2 Shows something like Figure 1 1. An exemplary imaging system 200 of the CT system 100 of FIG. According to aspects of the present disclosure, the imaging system 200 is configured to image a subject 204 (eg, Figure 1In one embodiment, the imaging system 200 includes a detector array 108 (see Figure 1 ). The detector array 108 also includes a plurality of detector elements 202 that together sense the X-ray radiation beam 106 that passes through a subject 204 (such as a patient) (see Figure 2 ) to collect corresponding projection data. Therefore, in one embodiment, the detector array 108 is manufactured in a multi-row or row configuration including multiple rows or rows of units or detector elements 202. In such a configuration (e.g., a multi-row or row detector CT or MDCT), another row or rows of detector elements 202 are arranged in a parallel configuration for acquiring projection data. The configuration may include 4, 8, 16, 32, 64, 128, or 256 rows or rows of detector elements. For example, a 64-row MDCT scanner may have 64 rows or rows of detector elements, while a 256-row MDCT scanner may have 256 rows or rows of detector elements. Therefore, four rotations of a spiral scan performed by a 64-row or row MDCT scanner may achieve detector coverage equal to a single rotation of a scan performed by a 256-row or row MDCT scanner.

[0051] In certain embodiments, the imaging system 200 is configured to traverse different angular positions around the subject 204 to collect the desired projection data. Thus, the gantry 102 and components mounted thereon can be configured to rotate about the rotation center 206 to collect, for example, projection data at different energy levels. Alternatively, in embodiments where the projection angle relative to the subject 204 changes over time, the mounted components can be configured to move along a generally curved line rather than along a section of a circle.

[0052] Thus, as the X-ray source 104 and the detector array 108 rotate, the detector array 108 collects data of the attenuated X-ray beam. The data collected by the detector array 108 then undergoes preprocessing and calibration to condition the data to represent the line integral of the attenuation coefficient of the scanned subject 204. The processed data is often referred to as a projection.

[0053] In some examples, individual detectors or detector elements 202 in detector array 108 may include photon counting detectors that register interactions of individual photons into one or more energy bins. It should be appreciated that the methods described herein may also be implemented using energy integrating detectors.

[0054] The acquired projection data sets may be used for basis material decomposition (BMD). During BMD, the measured projections are converted into a set of material density projections. The material density projections may be reconstructed to form a pair or set of material density maps or images for each corresponding basis material (such as bone, soft tissue, and / or contrast agent maps). The density maps or images may then be correlated to form a 3D volumetric image of the basis material (e.g., bone, soft tissue, and / or contrast agent) in the imaging volume.

[0055] Once reconstructed, the base material image produced by the imaging system 200 shows the internal features of the subject 204 represented by the density of the two base materials. The density image can be displayed to show these features. In traditional methods of diagnosing medical conditions (such as disease states), and more generally diagnosing medical events, a radiologist or physician will consider a hard copy or display of a density image to discern characteristic features of interest. Such features can include lesions, size and shape of specific anatomical structures or organs, as well as other features that should be discernible in the image based on the skill and knowledge of the individual practitioner.

[0056] In one embodiment, the imaging system 200 includes a control mechanism 208 to control movement of components, such as rotation of the gantry 102 and operation of the x-ray source 104. In certain embodiments, the control mechanism 208 also includes an x-ray controller 210 configured to provide power and timing signals to the x-ray source 104. Additionally, the control mechanism 208 includes a gantry motor controller 212 configured to control the rotational speed and / or position of the gantry 102 based on imaging requirements.

[0057] In certain embodiments, the control mechanism 208 also includes a data acquisition system (DAS) 214 that is configured to sample analog data received from the detector elements 202 and convert the analog data into digital signals for subsequent processing. The DAS 214 can also be configured to selectively aggregate analog data from a subset of the detector elements 202 into a so-called macro-detector, as further described herein. The data sampled and digitized by the DAS 214 is transmitted to a computer or computing device 216. In one example, the computing device 216 stores the data in a storage device or mass storage device 218. For example, the storage device 218 can include a hard drive, a floppy disk drive, a compact disk-read / write (CD-R / W) drive, a digital versatile disk (DVD) drive, a flash drive, and / or a solid-state storage drive.

[0058] In addition, the computing device 216 provides commands and parameters to one or more of the DAS 214, the X-ray controller 210, and the gantry motor controller 212 to control system operations, such as data acquisition and / or processing. In certain embodiments, the computing device 216 controls system operations based on operator input. The computing device 216 receives operator input, which includes, for example, commands and / or scan parameters, via an operator console 220 operatively coupled to the computing device 216. The operator console 220 may include a keyboard (not shown) or a touch screen to allow an operator to specify commands and / or scan parameters.

[0059] Although Figure 2 One operator console 220 is shown, but more than one operator console may be coupled to the imaging system 200, e.g., for inputting or outputting system parameters, requesting exams, plotting data, and / or viewing images. In addition, in certain embodiments, the imaging system 200 may be coupled to multiple displays, printers, workstations, and / or the like located locally or remotely, e.g., within an institution or hospital or at an entirely different location, via one or more configurable wired and / or wireless networks (such as the Internet and / or a virtual private network, a wireless telephone network, a wireless local area network, a wired local area network, a wireless wide area network, a wired wide area network, etc.).

[0060] In one embodiment, for example, the imaging system 200 includes or is coupled to a picture archiving and communication system (PACS) 224. In an exemplary implementation, the PACS 224 is further coupled to a remote system (such as a radiology department information system, a hospital information system) and / or to an internal or external network (not shown) to allow operators at different locations to supply commands and parameters and / or gain access to image data.

[0061] The computing device 216 uses operator-provided and / or system-defined commands and parameters to operate a table motor controller 226, which in turn can control the table 114, which can be a motorized table. Specifically, the table motor controller 226 can move the table 114 to properly position the subject 204 in the gantry 102 to acquire projection data corresponding to a target volume of the subject 204.

[0062] As previously described, the DAS 214 samples and digitizes the projection data collected by the detector elements 202. Subsequently, the image reconstructor 230 uses the sampled and digitized X-ray data to perform high-speed reconstruction. Figure 2The image reconstructor 230 is shown as a separate entity, but in certain embodiments, the image reconstructor 230 may form part of the computing device 216. Alternatively, the image reconstructor 230 may not be present in the imaging system 200, and instead the computing device 216 may perform one or more functions of the image reconstructor 230. Furthermore, the image reconstructor 230 may be located locally or remotely, and may be operably connected to the imaging system 200 using a wired or wireless network. In particular, one exemplary embodiment may use computing resources in a "cloud" network cluster for the image reconstructor 230.

[0063] In one embodiment, the image reconstructor 230 stores the reconstructed image in the storage device 218. Alternatively, the image reconstructor 230 may transmit the reconstructed image to the computing device 216 to generate usable patient information for diagnosis and evaluation. In certain embodiments, the computing device 216 may transmit the reconstructed image and / or the patient information to a display or display device 232 that is communicatively coupled to the computing device 216 and / or the image reconstructor 230. In some embodiments, the reconstructed image may be transmitted from the computing device 216 or the image reconstructor 230 to the storage device 218 for short-term or long-term storage.

[0064] Figure 3 FIG. 4 shows a schematic diagram of a CT system when detecting a patient. Figure 3As shown, the CT system 310 generally includes a rotatable gantry 312 and a support table 315 arranged in a hollow imaging area 314 of the rotatable gantry 312 for carrying a patient 330. The rotatable gantry 312 includes an X-ray source S and a detector 318 arranged opposite to the X-ray source S, wherein the detector 318 includes a plurality of independent detector units D arranged in an array. When the rotatable gantry 312 is located at a certain scanning position, the X-ray source S emits a fan-shaped X-ray beam 320 in the direction of the detector 318, and the plurality of detector units D respectively sense the X-rays attenuated by the patient 330, so that a group of projection data is sensed by the detector units D, and a corresponding frame of projection data is obtained. As the rotatable gantry 312 rotates, the X-ray source S and the detector 318 rotate around the rotation center O, and the CT system 310 performs multiple scans. In each scanning process, all the detector units D can sense and obtain each corresponding frame of projection data. When the detector units D are normal, the corresponding frame of projection data can be directly used to reconstruct one or more images. The direction of the detector 318 in which the inspection object moves toward or out of the medical imaging system is called the row direction, that is, the direction in which the inspection object on the support table 315 moves toward or out of the rotatable rack 312. The extending direction of the detector 318 that is perpendicular to the row direction and arranged along a part surrounding the inspection object is called the channel direction, that is, the direction in which the detector 318 is arranged in an arc along the rotatable rack 312. The angle at which the detector 318 acquires raw projection data at different positions surrounding the inspection object is called the viewing angle direction, that is, the different angles of the detector 318 when it surrounds the inspection object along the rotatable rack 312.

[0065] In order to obtain higher resolution reconstructed images, a new technology has emerged in recent years, in which the imaging process is improved by artificial intelligence. At present, the main improvement method is to obtain high-resolution images based on low-resolution images. Although this image-to-image method is relatively simple and intuitive, due to the pixel-to-pixel and the uninterpretability of deep learning networks, defects such as false structures, bridging or edge overshoot may occur in the image, which is a major problem for medical imaging. Another improvement method is to make predictions based on projection data. However, since projection data is the basis of image reconstruction, if the prediction of projection data is wrong, it is easy to produce stripe defects in the reconstructed image. Therefore, it is very difficult to predict and obtain high-resolution images based on projection data.

[0066] In view of the above problems, the embodiments of the present disclosure innovatively propose an image processing method and system, a neural network model training method and a medical imaging system that utilize a neural network model and projection data to improve image resolution.

[0067] Figure 4A flow chart of an image processing method 400 according to an embodiment of the present disclosure is shown. In step 402, raw projection data of an object under examination is acquired, wherein the raw projection data is obtained by scanning the object under examination by a medical imaging system. Next, in step 404, input features are constructed, wherein the input features include trend information of the raw projection data. Then, in step 406, a neural network model is used to generate enhanced projection data of the object under examination based on the input features, wherein the enhanced projection data has a higher resolution than the raw projection data and is used to reconstruct a medical image of the object under examination.

[0068] Figure 5 A schematic diagram of raw projection data according to an embodiment of the present disclosure is shown, where the raw projection data is three-dimensional projection data acquired by a detector of a medical imaging system. For example, the raw projection data is obtained by the aforementioned Figures 1 to 3 The detectors 108, 318 of the CT systems 100, 200, 310 described herein acquire. In one embodiment, the medical imaging system may be a computed tomography (CT) medical imaging system, a positron emission tomography (PET-CT) medical imaging system, or a positron emission tomography (PET) medical imaging system. The original projection data includes three dimensions: a row direction (Z direction), a channel direction (X direction), and a viewing angle direction (Y direction). The row direction (Z direction) indicates the direction in which the detector moves toward or out of the medical imaging system when the object under examination moves, that is, the scanning translation direction of the medical imaging system. For example, the detectors 108, 318 of the CT systems 100, 200, 310 may be configured with different numbers of rows of detector units in the row direction (Z direction), for example, 8 rows, 16 rows, 32 rows, 64 rows, 256 rows, 512 rows, and the like. The channel direction (X direction) indicates the extending direction of the detectors arranged perpendicular to the row direction and partially around the examination object, that is, the width direction of the detectors 108, 318 of the medical imaging system. For example, there may be about 900 channels. The viewing angle direction (Y direction) indicates the angle at which the detector acquires raw projection data at different positions around the examination object. For example, the viewing angle direction (Y direction) indicates the angle at which the detector acquires raw projection data at different positions around the examination object. Figures 1 to 3 The described CT systems 100, 200, 310 acquire views at angles or viewing angles at which the detectors 108, 318 are rotated with the gantry. For example, in an axial scan, the medical imaging system may acquire raw projection data or views at approximately 1000 angular positions, each of which is referred to as a viewing angle.

[0069] Trend information refers to the changing trend of information presented or contained in the original projection data or the data obtained after processing the original projection data. The trend information of the original projection data may include information about the structure or attributes of the inspection object covered by the original projection data itself acquired by the detector of the medical imaging system (e.g., the detector 108, 318 of the CT system 100, 200, 310). For example, since the tissue information in the organs of the human body is constantly changing and correlated between different positions, after scanning the organs of the human body, the data points in the acquired original projection data have a changing trend in the row direction, channel direction and viewing angle direction, that is, the information presented or contained in the projection data itself.

[0070] In addition to the projection data itself, the difference between the projection data of two adjacent viewing positions in the viewing direction contains or presents the structural difference information of the inspection object at the adjacent viewing positions, such as the residual boundary of the anatomical tissue presented in the projection data. Therefore, the difference data between the projection data of two adjacent viewing positions in the viewing direction can also reflect trend information.

[0071] The trend information of the original projection data may also include the trend information presented by the data after the original projection data is processed. For example, in CT image processing, the kernel function or convolution kernel is an important parameter for image reconstruction, which mainly affects the sharpness and noise level of the image by adjusting the frequency content of the projection data. Different types of kernel functions can be used for different anatomical structures. For example, the bone kernel can improve the spatial resolution of the bone, while the soft tissue kernel is suitable for soft tissue imaging. In a tomographic image acquired by a medical imaging system, different soft tissues and high-frequency tissues usually exist at the same time, such as lungs and vertebrae, heart and vertebrae, liver and vertebrae, brain soft tissue and skull, etc. Different tissues correspond to different kernel functions, and different kernel functions have different cutoff frequencies and enhancement functions. If the operator wants to clearly see the different frequency information of different organs or tissues, then it is necessary to filter the projection data with different kernel functions to obtain projection data with different frequency enhancements. Therefore, the original projection data is filtered using kernel functions with different frequencies, and the filtered data corresponding to different kernel functions can reflect the frequency change trend information, that is, the frequency trend information.

[0072] In addition, the trend information of the raw projection data may also include the input order when the acquired trend information is input into the neural network model in a specific order. For example, when the raw projection data is filtered using kernel functions with different frequencies, the filtered data may be input into the neural network model in the order of increasing frequency or decreasing frequency, so this processing method also reflects the sequential trend information.

[0073] The present disclosure innovatively proposes to use the trend information of projection data to construct input features to improve the resolution of the original projection data. Usually, a large number of projection data points need to be inferred and processed during image reconstruction. Even if only one inference error occurs, it will cause easily identifiable defects such as stripes in the reconstructed image. Therefore, when using projection data for image enhancement, higher accuracy of inference is required. The present disclosure uses one or more trend information to make more accurate predictions when the neural network model makes predictions, by drawing on the trend information of the projection data itself and / or the trend information presented by the data after processing the original projection data.

[0074] In one embodiment, in step 404, constructing the input feature may include: constructing the trend information of the original projection data as an input channel of the input feature. It should be understood that the "input channel" of the input feature corresponds to the concept of describing the dimension of the feature in artificial intelligence, and the "channel direction" of the projection data corresponds to the extension direction of the detector of the medical imaging system arranged along a local surrounding of the inspection object.

[0075] In one embodiment, the trend information of the original projection data may include projection data trend information in at least one dimension.

[0076] In one embodiment, the trend information of the original projection data may include the projection data trend information in the row direction and the channel direction. As an example, when constructing the input feature based on the projection data trend information in the row direction (Z direction) and the channel direction (X direction), the original projection data in the plane formed by the row direction and the channel direction at a viewing angle position in the viewing angle direction may be constructed as an input channel of the input feature. Figure 5 The data block corresponding to the top box shown is constructed as an input channel of the input feature.

[0077] In one embodiment, the trend information of the original projection data may include the projection data trend information in the viewing angle direction. For example, the trend information of the original projection data may include the projection data trend information of the difference between the projection data block of the original projection data at a viewing angle position and the projection data block at an adjacent viewing angle position. As an example, when constructing the input feature based on the trend information in the viewing angle direction (Y direction), the difference between the data block in the plane formed by the row direction and the channel direction at a viewing angle position of the original projection data in the viewing angle direction and the data block in the plane formed by the row direction and the channel direction at an adjacent viewing angle position in the viewing angle direction can be calculated, and the difference is constructed as an input channel of the input feature. Figure 6A schematic diagram of obtaining projection data trend information in the viewing angle direction of original projection data according to an embodiment of the present disclosure is shown. For the data block corresponding to the solid line box at the top, the difference between the data block and the data block at the adjacent viewing angle position represented by the dotted line box can be calculated. Therefore, the difference can be constructed as an input channel of the input feature.

[0078] In one embodiment, the trend information of the original projection data may include frequency trend information in a specific order obtained by filtering the original projection data of at least one position in at least one dimension using at least two kernel functions with different frequencies. Figure 7 A schematic diagram of the frequency trend information of the original projection data according to an embodiment of the present disclosure is shown. In a tomographic image acquired by a medical imaging system, different soft tissues and high-frequency tissues usually exist at the same time, for example, lungs and vertebrae, heart and vertebrae, liver and vertebrae, brain soft tissue and skull, etc. Different tissues correspond to different kernel functions, and different kernel functions have different cutoff frequencies and enhancement functions. If the operator wants to clearly see the different frequency information of different organs or tissues, it is necessary to filter the projection data with different kernel functions to obtain projection data with different frequency enhancements. Therefore, the projection data enhanced with different frequencies can also reflect the changing trend of the clarity of the data. As shown in the figure, for the data block in the plane formed by the row direction and the channel direction at a viewing angle position of the original projection data in the viewing angle direction, the data block can be filtered using kernel function 1, kernel function 2, and kernel function 3 with different frequencies respectively. If there is a tissue corresponding to the high frequency in the data block, the tissue will produce a gradually changing clarity after being processed by kernel functions of different frequencies. As an example, when constructing input features based on frequency trend information, the filtered data blocks obtained by filtering the data blocks in the plane formed by the row direction and the channel direction at a viewing position in the viewing direction of the original projection data using kernel functions with different frequencies can be respectively constructed as an input channel of the input feature according to the frequency change order of the kernel function. For example, when there are three kernel functions, the first filtered data block, the second filtered data block, and the third filtered data block can be constructed as three input channels of the input feature in the order of increasing or decreasing cutoff frequencies of the three kernel functions, for example, the first filtered data block obtained by filtering with the first kernel function with the highest cutoff frequency is used as the first input channel of the input feature, the second filtered data block obtained by filtering with the second kernel function with a medium cutoff frequency is used as the second input channel of the input feature, and the third filtered data block obtained by filtering with the third kernel function with the lowest cutoff frequency is used as the third input channel of the input feature.

[0079] In one embodiment, the trend information of the original projection data may include the projection data trend information presented by the projection data block at at least one position in at least one dimension and the frequency trend information presented by the filtered projection data block obtained by filtering the projection data block by at least two kernel functions of different frequencies. As an example, when constructing input features based on the projection data trend information and the frequency trend information, the projection data trend information and the frequency trend information may be constructed as input channels of the input features respectively. For example, when the trend information of the original projection data includes Figure 5 The projected data trend information in the row direction and channel direction shown, Figure 6 The projection data trend information in the viewing direction shown, and Figure 7 When the frequency trend information is shown, the projection data trend information in the row direction and the channel direction can be constructed as the first input channel of the input feature, the projection data trend information in the viewing angle direction can be constructed as the second input channel of the input feature, and the frequency trend information can be constructed as the third input channel, the fourth input channel, and the fifth input channel of the input feature in the order of increasing or decreasing cutoff frequency of the kernel function. It should be understood that the above is only an example of constructing the input feature, and one or more of the above trend information can be used to construct the input feature as needed, and the position of different trend information in the input feature can be adjusted as needed.

[0080] In one embodiment, the projection data block may include the original projection data in a plane formed by the other two dimensions at a position of the original projection data in one dimension of at least one dimension. For example, the projection data block may include the original projection data in a plane formed by the row direction and the channel direction at a viewing angle position in the viewing angle direction. Additionally or alternatively, the projection data block may include the original projection data in a plane formed by the viewing angle direction and the channel direction at a row position in the row direction. Additionally or alternatively, the projection data block may include the original projection data in a plane formed by the row direction and the viewing angle direction at a channel position in the channel direction.

[0081] Therefore, the present disclosure fully mines one or more trend information based on projection data and utilizes the inherent laws of various trend information to achieve better image resolution improvement.

[0082] Figure 8 The method for batch processing in a channel-row plane (i.e., within a certain viewing angle range) according to an embodiment of the present disclosure is shown. Since there is an upper limit to the size of data that can be processed each time during the processing of image data, it is necessary to extract multiple smaller-sized data blocks from the complete larger-sized three-dimensional projection data obtained by the detector of the medical imaging system for the inspection object, and then process each smaller-sized data block in turn. For example, Figure 8As shown, the projection data on the right side corresponds to the box on the top left side, that is, it corresponds to a certain viewing angle. First, take out the first data block corresponding to the black solid line box, and refer to Figure 4 The method 400 processes the first data block. Then, the second data block corresponding to the black dotted box is taken out, and the Figure 4 The method 400 processes the second data block. Next, take out the third data block corresponding to the black dotted line box, and refer to Figure 4 The method 400 processes the third data block, and so on, until the traversal Figure 8 All projection data on the right. The size of the data block can be adjusted as needed. The first interval between the first and second data blocks and the second interval between the second and third data blocks can be the same or different, and the first interval and the second interval can be adjusted as needed. Figure 8 The first and second data blocks are shown to overlap, and the second and third data blocks are shown to overlap, but the first, second, and third data blocks may also not overlap each other. Figure 8 The entire range on the right side can fully mine the trend information in the channel-row plane under the viewing angle. In addition, the data in the row-channel plane corresponding to each viewing angle of the original projection data can be processed in the same way to fully mine the trend information of the entire original projection data. It should be understood that although Figure 8 Only batch processing within the channel-row plane is shown, but the method can also be applied to the row-view plane and the channel-view plane.

[0083] Fig. 9 A flowchart of a method 900 for training a neural network model according to an embodiment of the present disclosure is shown.

[0084] In step 902, a training data set is acquired, the training data set including training original projection data and training enhanced projection data. The training original projection data and the training enhanced projection data can each be used to reconstruct a medical image of the inspection object. The training enhanced projection data has a higher resolution than the training original projection data, and the training enhanced projection data is used as the true value of the output of the neural network model.

[0085] The training data set can be obtained from a reconstructed image with high resolution, such as a high-resolution reconstructed image obtained by micro computed tomography (Micro-CT), and the high-resolution reconstructed image is converted into projection data, that is, high-resolution projection data, and then the high-resolution projection data is downsampled to obtain low-resolution projection data. Thus, the low-resolution projection data can be used as the training original data in the training data set, and the high-resolution projection data can be used as the training enhanced projection data in the training data set.

[0086] In step 904, training input features are constructed, where the training input features include trend information of the training original projection data.

[0087] In one embodiment, the training raw projection data is three-dimensional projection data acquired by a detector of a medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector when the inspection object moves toward or out of the medical imaging system, the channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object, and the viewing angle direction indicates the angle at which the detector acquires the training raw projection data at different positions around the inspection object.

[0088] In one embodiment, the trend information of the training raw projection data includes one or more of the following: projection data trend information in at least one dimension; and frequency trend information in a specific order obtained by filtering the training raw projection data of at least one position in at least one dimension using at least two kernel functions with different frequencies.

[0089] In one embodiment, at step 904, constructing the training input feature includes: constructing the trend information of the training original projection data as the input channel of the training input feature.

[0090] The process of constructing the training input features in step 904 is similar to the process of constructing the input features in step 404. To avoid redundancy, the specific details are not described here.

[0091] At step 906, a neural network model is used to generate prediction results of enhanced projection data having a higher resolution than the training original projection data based on the training input features. The neural network model may adopt any suitable resolution enhancement model.

[0092] In step 908, the loss function between the predicted result and the true value is calculated.

[0093] In one embodiment, the loss function includes at least one of the following: mean absolute error loss function, average structural similarity index metric loss function, perceptual loss function. A weighted sum of different loss functions can be used as the total loss function.

[0094] Mean absolute error loss function Loss MAE It can be used to calculate the loss pixel by pixel, and the calculation formula is as follows:

[0095] Loss MAE =|I predict -I gt |

[0096] I predict Represents the value of the predicted result of the pixel, and Igt represents the true value of the pixel.

[0097] Average Structural Similarity Index Measure (SSIM) Loss Function SSIM It can be used to calculate the similarity of each block. Due to the high requirements of medical images for structural authenticity, it is necessary not only to ensure that the visual clarity of the image is improved, but also to ensure that the tissue structure in the image remains unchanged. Therefore, this loss function helps to ensure the consistency between the image and the real tissue structure. The calculation formula is as follows:

[0098]

[0099] Among them, μ predict and μ gt are the means of the predicted results and the true values, respectively. and are the variances of the predicted results and the true values, σ predict，gt is the covariance between the predicted result and the true value, c1 and c2 are small constants.

[0100] Perceptual loss function Loss Perceptual It can be used to evaluate semantic similarity. Since this loss function can evaluate the similarity of high-level feature domains, it can improve robustness in denoising. For example, the VGG19 relu5_4 layer can be used, and the calculation formula is as follows:

[0101] Loss Perceptual =||vgg19(I predict )-vgg19(I gt )||2

[0102] The present disclosure can further improve the structural similarity and / or semantic similarity by improving the loss function, thereby enhancing the authenticity and robustness of the resolution enhancement results.

[0103] In step 910, the parameters of the neural network model are updated based on the loss function to obtain a trained neural network model.

[0104] Therefore, the present disclosure trains a neural network model based on one or more trend information of projection data so that the neural network model can achieve better image resolution improvement.

[0105] Fig.10 A flow chart of a method 1000 for image processing according to another embodiment of the present disclosure is shown. In order to improve the resolution of image data, the present disclosure also proposes a method 1000, which adopts an improved neural network model, and the structure of the neural network model is described in detail later. In step 1002, original projection data is acquired. The original projection data is obtained by scanning the inspection object by a medical imaging system. Next, in step 1004, input features are constructed, and the input features include trend information of the original projection data. Then, in step 1006, a neural network model is used to generate enhanced projection data based on the input features, wherein the enhanced image data has a higher resolution than the original image data.

[0106] Fig.11A A schematic diagram of a neural network model 1100 according to an embodiment of the present disclosure is shown. In one embodiment, the neural network model 1100 may adopt a residual channel attention network (RCAN). The neural network model 1100 may receive trend information of raw projection data obtained by scanning an object under examination by a medical imaging system as input features 1102 and output enhanced projection data, wherein the enhanced projection data has a higher resolution than the raw projection data. The neural network model 1100 includes a shallow feature extraction layer 1110, a deep feature extraction layer 1120, and an upsampling layer 1130. The shallow feature extraction layer 1110 may include a convolution layer, for example, a two-dimensional convolution layer. The shallow feature extraction layer 1110 is used to perform feature extraction on the input feature 1102 to obtain shallow features. The deep feature extraction layer 1120 may include at least one cascaded residual group 1140A, 1140B, ... 1140N, a convolution layer 1124, and a summation module 1126, for extracting the shallow features from the shallow feature extraction layer 1110 to obtain deep features. The upsampling layer 1130 may upsample the deep features from the deep feature extraction layer 1120 to obtain output features 1104 of a desired resolution as enhanced projection data. For example, the upsampling layer 1130 may include a pixel shuffling layer and a convolution layer.

[0107] In one embodiment, the original projection data is three-dimensional projection data acquired by a detector of a medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector when the inspection object moves toward or out of the medical imaging system. The channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object. The viewing angle direction indicates the angle at which the detector acquires the original projection data at different positions around the inspection object.

[0108] In one embodiment, the trend information of the original projection data includes one or more of the following: projection data trend information in at least one dimension; and frequency trend information in a specific order obtained by filtering the original projection data of at least one position in at least one dimension using at least two kernel functions with different frequencies.

[0109] Fig. 11B A schematic diagram of a residual group according to an embodiment of the present disclosure is shown. The residual group may include a plurality of cascaded residual blocks 1150A, 1150B, ... 1150N, a splicing layer 1142 and a convolution layer 1144. Each residual block 1150A, 1150B, ... 1150N can extract depth features of different levels from the input of the residual group. The splicing layer 1142 can splice the depth features extracted by each residual block to obtain the spliced ​​depth features. The convolution layer 1144 can perform a convolution operation on the spliced ​​depth features to obtain the output of the residual group. The splicing layer structure does not exist in the conventional residual group structure of the RCAN network. The present disclosure innovatively introduces a splicing layer after the residual block, which can splice the depth features of different levels and realize the fusion of different depth features, thereby retaining the information details of different depths and improving the resolution of the output image.

[0110] Fig. 11CA schematic diagram of a residual block according to an embodiment of the present disclosure is shown. The residual block includes a plurality of parallel convolutional layers 1152, 1154, 1156 of different sizes, activation function modules 1158A, 1158B, 1158C each cascaded with a convolutional layer, a concatenation layer 1160, a final convolutional layer 1162, and a summation module 1164. Each convolutional layer 1152, 1154, 1156 is used to perform a convolution operation on the input of the residual block to obtain a convolution result. The convolutional layers 1152, 1154, 1156 use convolutional kernels of different sizes, for example, the convolutional layer 1152 uses a convolutional kernel of size 1x1, the convolutional layer 1154 uses a convolutional kernel of size 3x3, and the convolutional layer 1156 uses a convolutional kernel of size 5x5, thereby achieving multi-scale feature extraction by introducing different receptive field sizes. The activation function modules 1158A, 1158B, and 1158C are each cascaded with one of the convolutional layers 1152, 1154, and 1156 to apply the activation function to the convolution result of the corresponding convolutional layer to obtain local features. The activation function modules 1158A, 1158B, and 1158C can use a leaky linear rectifier function (LeakyReLU) layer. A convolutional layer and an activation function module constitute a sub-branch of the residual block, wherein the convolutional layer 1152 and the activation function module 1158A constitute the first sub-branch, the convolutional layer 1154 and the activation function module 1158B constitute the second sub-branch, and the convolutional layer 1156 and the activation function module 1158C constitute the third sub-branch. Although Fig. 11C Only three sub-branches are shown. It should be understood that the number of sub-branches can be set as needed, and the size of the convolution layer can also be selected as needed, without being limited to Fig. 11C The concatenation layer 1160 may concatenate the local features generated by the activation function modules 1158A, 1158B, and 1158C to obtain concatenated local features. The final convolution layer 1162 may perform a convolution operation on the concatenated local features to obtain a final convolution result. The summation module 1164 may add the final convolution result to the input of the residual block to obtain the output of the residual block.

[0111] Therefore, the neural network model proposed in the present disclosure can fuse different depth features of the projection data, so that the output projection data has richer details and improved resolution.

[0112] In one embodiment, the neural network model 1100 can also be applied to the neural network model in method 400 .

[0113] The neural network model 1100 can be trained by the following steps. First, a training data set is obtained, the training data set includes training original projection data and training enhanced projection data, wherein the training enhanced projection data has a higher resolution than the training original projection data, and the training enhanced projection data is used as the true value of the output of the neural network model. Secondly, a training input feature is constructed based on the trend information of the training original projection data. Next, a neural network model is used to generate a prediction result of the enhanced projection data having a higher resolution than the training original projection data based on the training input feature. Then, a loss function between the prediction result and the true value is calculated. Finally, the parameters of the neural network model are updated based on the loss function to obtain a trained neural network model.

[0114] In one embodiment, the loss function includes at least one of the following: mean absolute error loss function, average structural similarity index metric loss function, perceptual loss function. A weighted sum of different loss functions can be used as the total loss function.

[0115] In addition, the present disclosure further provides a medical imaging system, including: a scanning device configured to acquire raw projection data of an inspection object; and a processor configured to execute any one of the methods 400 , 900 , and 1000 .

[0116] In addition, the present disclosure also provides a non-transitory computer-readable medium having instructions stored thereon, where the instructions can be executed by a processor to implement any one of the methods 400 , 900 , and 1000 .

[0117] Fig. 12A and Fig. 12C shows a reconstructed image generated from the original projection data, Fig. 12B and Fig.12D A reconstructed image generated from enhanced projection data obtained using the present disclosure is shown. Fig. 12A and Fig. 12B The comparison and Fig. 12C and Fig.12D It can be seen from the comparison that according to the enhanced projection data obtained by using the present disclosure, the reconstructed image generated has higher clarity and does not have defects such as artifacts.

[0118] Fig.13 An example block diagram of a computing device 1300 according to an embodiment of the present disclosure is shown. The computing device 1300 may be used as Figure 2An example implementation of the computing device 216 is shown. The computing device 1300 includes: one or more processors 1320; a storage device 1310 for storing one or more programs, which, when executed by the one or more processors 1320, causes the one or more processors 1320 to implement the processes described in the present disclosure. The processor is, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0119] Fig.13 The computing device 1300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0120] like Fig.13 As shown, computing device 1300 is in the form of a general computing device. Components of computing device 1300 may include, but are not limited to: one or more processors 1320, storage device 1310, and bus 1350 connecting different system components (including storage device 1310 and processor 1320).

[0121] Bus 1350 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0122] The computing device 1300 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 1300, including volatile and non-volatile media, removable and non-removable media.

[0123] Storage device 1310 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1311 and / or cache memory 1312. Computing device 1300 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 1313 may be used to read and write non-removable, non-volatile magnetic media ( Fig.13 not shown, usually called a "hard drive"). Although Fig.13Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 1350 via one or more data medium interfaces. The storage device 1310 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0124] A program / utility 1314 having a set (at least one) of program modules 1315 may be stored, for example, in a storage device 1310, such program modules 1315 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 1315 generally perform the functions and / or methods of any embodiment described herein.

[0125] The computing device 1300 may also communicate with one or more external devices 1360 (e.g., keyboard, pointing device, display 1370, etc.), one or more devices that enable a user to interact with the computing device 1300, and / or any device that enables the computing device 1300 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through input / output (I / O) interface 1330. Furthermore, the computing device 1300 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) through network adapter 1340. Fig.13 As shown, the network adapter 1340 communicates with other modules of the computing device 1300 via the bus 1350. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the computing device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0126] The processor 1320 executes the programs stored in the storage device 1310 to perform various functional applications and data processing, such as implementing the process described in the present disclosure.

[0127] The technology described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any features described as modules or components can also be implemented together in an integrated logic device, or separately implemented as discrete but interoperable logic devices. If implemented in software, the technology can be implemented at least in part by a non-transient processor-readable storage medium including instructions, and when the instructions are executed, one or more of the above methods are executed. The non-transient processor-readable data storage medium can form a part of a computer program product that may include packaging materials. The program code can be implemented in a high-level procedural programming language or an object-oriented programming language to communicate with a processing system. If necessary, the program code can also be implemented in assembly language or machine language. In fact, the mechanism described herein is not limited to the scope of any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0128] One or more aspects of at least some embodiments may be implemented by representative instructions stored on a machine-readable medium that represent various logic within a processor, which when read by a machine causes the machine to fabricate logic for performing the techniques described herein.

[0129] Such machine-readable storage media may include, but are not limited to, a non-transitory tangible arrangement of articles manufactured or formed by a machine or apparatus, including storage media such as: a hard disk; any other type of disk, including floppy disks, optical disks, compact disk-read only memory (CD-ROM), compact disk-rewritable (CD-RW), and magneto-optical disks; semiconductor devices, such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); magnetic or optical cards; or any other type of medium suitable for storing electronic instructions.

[0130] Instructions may also be further sent or received via a communication network using a transmission medium via a network interface device that utilizes any of a number of transmission protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.).

[0131] Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, and a wireless data network (e.g., a wireless network called The Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards, known as ), IEEE 802.15.4 series of standards, peer-to-peer (P2P) networks, etc. In an example, the network interface device may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connecting to a communication network. In an example, the network interface device may include multiple antennas that communicate wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies.

[0132] The term "transmission medium" shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine and includes digital or analog communications signals or other intangible medium for facilitating communication of such software.

[0133] So far, the image processing method and system, the neural network model training method and the medical imaging system according to the present invention are described, and a computer-readable storage medium capable of implementing the method is also described.

[0134] Some exemplary embodiments have been described above. However, it should be understood that various modifications may be made to the exemplary embodiments described above without departing from the spirit and scope of the present invention. For example, if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in different ways and / or replaced or supplemented by other components or their equivalents, suitable results may be achieved, and accordingly, these modified other implementations also fall within the scope of protection of the claims.

Claims

1. A method for image processing, comprising: Acquiring original projection data of an examination object, wherein the original projection data is obtained by scanning the examination object with a medical imaging system; constructing input features, wherein the input features include trend information of the original projection data; as well as A neural network model is used to generate enhanced projection data of the examination object based on the input features, wherein the enhanced projection data has a higher resolution than the original projection data and is used to reconstruct a medical image of the examination object.

2. The method according to claim 1, characterized in that The original projection data is three-dimensional projection data acquired by the detector of the medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector and the direction in which the inspection object moves toward or out of the medical imaging system. The channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object. The viewing angle direction indicates the angle at which the detector acquires the original projection data at different positions around the inspection object.

3. The method according to claim 2, characterized in that The trend information of the raw projection data includes one or more of the following: Projected data trend information in at least one dimension; and The frequency trend information in a specific order is obtained by filtering the original projection data of at least one position in the at least one dimension using at least two kernel functions with different frequencies.

4. The method according to claim 3, characterized in that The trend information of the original projection data includes projection data trend information presented by the projection data block at at least one position in the at least one dimension and frequency trend information presented by the filtered projection data block obtained by filtering the projection data block by at least two kernel functions of different frequencies.

5. The method according to claim 4, characterized in that The projection data block includes raw projection data within a plane formed by the other two dimensions at a position of the raw projection data in one dimension of the at least one dimension.

6. The method according to claim 1, characterized in that Constructing input features includes: The trend information of the original projection data is constructed as an input channel of the input feature.

7. The method according to claim 2, characterized in that The medical imaging system is a computed tomography medical imaging system, a positron emission tomography medical imaging system, or a positron emission scanning medical imaging system.

8. A method for training a neural network model, comprising: Acquire a training data set, the training data set comprising training original projection data and training enhanced projection data, wherein the training original projection data and the training enhanced projection data can each be used to reconstruct a medical image of an examination object, the training enhanced projection data has a higher resolution than the training original projection data, and the training enhanced projection data is used as a true value of an output of the neural network model; Constructing a training input feature, wherein the training input feature includes trend information of the training original projection data; Using the neural network model to generate prediction results of enhanced projection data having a higher resolution than the training original projection data based on the training input features; Calculating a loss function between the prediction result and the true value; as well as The parameters of the neural network model are updated based on the loss function to obtain a trained neural network model.

9. The method according to claim 8, characterized in that The training original projection data is three-dimensional projection data acquired by the detector of the medical imaging system and includes three dimensions: row direction, channel direction and viewing angle direction. The row direction indicates the direction of the detector and the direction in which the inspection object moves toward or out of the medical imaging system. The channel direction indicates the extension direction of the detector which is perpendicular to the row direction and arranged locally around the inspection object. The viewing angle direction indicates the angle at which the detector acquires the training original projection data at different positions around the inspection object.

10. The method according to claim 9, characterized in that The trend information of the training raw projection data includes one or more of the following: Projected data trend information in at least one dimension; and The frequency trend information in a specific order is obtained by filtering the training original projection data of at least one position in the at least one dimension using at least two kernel functions with different frequencies.

11. The method according to claim 8, characterized in that Constructing training input features includes: The trend information of the training original projection data is constructed as an input channel of the training input feature.

12. The method according to claim 8, characterized in that The loss function includes at least one of the following: a mean absolute error loss function, an average structural similarity index measurement loss function, and a perceptual loss function.

13. A system for image processing, comprising a neural network model, characterized in that: The neural network model receives trend information of original projection data acquired by scanning an examination object by a medical imaging system as an input feature and outputs enhanced projection data, wherein the enhanced projection data has a higher resolution than the original projection data; The neural network model includes: A shallow feature extraction layer, used for extracting features from the input features using a convolutional layer to obtain shallow features; A deep feature extraction layer, configured to extract the shallow features using at least one cascaded residual group, convolutional layer, and summation module to obtain deep features; and An upsampling layer is used to upsample the deep features into the enhanced projection data.

14. The system of claim 13, wherein: Each residual group of the deep feature extraction layer includes: A plurality of cascaded residual blocks, each residual block is used to extract deep features of different levels from the input of the residual group; A concatenation layer, used to concatenate the deep features extracted from each residual block to obtain a concatenated deep feature; and A convolution layer is used to perform a convolution operation on the concatenated deep features to obtain an output of the residual group.

15. The system of claim 14, wherein: The residual block comprises: A plurality of parallel convolutional layers of different sizes, each convolutional layer being used to perform a convolution operation on the input of the residual block to obtain a convolution result; A plurality of activation function modules, each activation function module is cascaded with one convolution layer of the plurality of parallel convolution layers of different sizes, and is used to apply an activation function to a convolution result of a corresponding convolution layer to obtain a local feature; A concatenation layer, used for concatenating the local features of each activation function module to obtain concatenated local features; A final convolution layer, configured to perform a convolution operation on the concatenated local features to obtain a final convolution result; and A summing module is used to add the final convolution result to the input of the residual block to obtain the output of the residual block.

16. The system of claim 13, wherein: The original projection data is three-dimensional projection data acquired by the detector of the medical imaging system and includes three dimensions: a row direction, a channel direction, and a viewing angle direction. The row direction indicates the direction of the detector and the direction in which the inspection object moves toward or out of the medical imaging system. The channel direction indicates the extension direction of the detector that is perpendicular to the row direction and arranged locally around the inspection object. The viewing angle direction indicates the angle at which the detector acquires the original projection data at different positions around the inspection object.

17. The system of claim 16, wherein: The trend information of the raw projection data includes one or more of the following: Projected data trend information in at least one dimension; and The frequency trend information in a specific order is obtained by filtering the original projection data of at least one position in the at least one dimension using at least two kernel functions with different frequencies.

18. A medical imaging system comprising: A scanning device configured to acquire raw projection data of an inspection object; as well as A processor configured to execute the method according to any one of claims 1 to 12.

19. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions are executable by a processor to implement the method according to any one of claims 1 to 12.

Images