Medical image synthesis apparatus and method

Abstract

Embodiments of the present application provide a medical image synthesis device and method. The method comprises: obtaining a first medical image and a second medical image; registering the first medical image and the second medical image; determining first parameter values of each pixel position on the registered first medical image and second parameter values of each pixel position on the second medical image; multiplying the first parameter values and the second parameter values on the same pixel positions of the registered first medical image and the second medical image, and generating a synthesis image data according to the multiplication result. Therefore, the multiplication of the first parameter values and the second parameter values is equivalent to amplifying the difference of pixel values of the positions of normal tissue and lesion tissue, so that the contrast and resolution of the synthesis image are higher, thereby making the lesion area clearer and improving the lesion detection efficiency.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and more particularly to a medical image synthesis apparatus and method. Background Technology

[0002] Currently, medical imaging equipment is increasingly widely used to scan subjects (e.g., the human body) to obtain medical images of specific areas (e.g., whole or partial organs, or specific regions of interest), providing useful information for medical diagnosis. Medical imaging scans include computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), single-photon emission computed tomography (SPECT), and so on. Among these, because different tissues or organs have different attenuation or absorption coefficients for X-rays, CT images are cross-sectional tomographic images of the human body, reflecting its anatomical structure. PET or SPECT images selectively reflect the metabolism, receptors, and gene expression of target tissue cells based on radioactive tracers, enabling early reflection of physiological, pathological, biochemical, and metabolic changes in human tissues at the molecular level. PET / CT and SPECT / CT images are imaging modalities that combine functional metabolic molecular imaging and anatomical structural imaging. PET / CT has unique value in the diagnosis of diseases such as tumors, cardiovascular and cerebrovascular diseases, neurodegenerative diseases, and epilepsy. SPECT / CT imaging is widely used in the diagnosis of various clinical diseases, including bone, heart, and tumors. MRI can reflect tissue structure and metabolism through changes in tissue cell T1 and T2 values ​​and proton distribution density. Summary of the Invention

[0003] Existing methods for fusing PET or SPECT images with CT images mainly include traditional overlay fusion methods and deep learning-based methods. The traditional overlay fusion method involves adjusting the CT image to a certain transparency level and then directly overlaying the PET or SPECT image onto the CT image to form a fused image. However, the inventors discovered that this method cannot accurately identify lesions in the fused image. For example, low contrast in the fused image may prevent accurate lesion identification, or small lesions in hollow organs containing tracers (such as the renal pelvis, blood vessel walls, and intestines) may not be accurately identified.

[0004] In addition, deep learning-based methods input PET or SPECT images and CT images into a trained convolutional neural network model, the output of which is the PET / CT image or SPECT / CT image. However, the inventors found that deep learning-based methods have the following problems: since the fused image has the same properties as the source image, the improvement in image resolution and contrast is limited; furthermore, deep learning-based methods are highly dependent on training data, and insufficient training data will affect the performance of the deep learning network, thus affecting the image fusion effect; and the resolution of the fused image is still relatively low, with poor detection efficiency for some small lesions and lesions in hollow organs. Because the standard uptake values ​​of images of normal tissues such as the kidneys, ureters, and bladder are too high, they may mask some tiny lesions, leading to missed diagnoses or misdiagnoses and reducing diagnostic accuracy.

[0005] To address at least one of the aforementioned technical problems, embodiments of this application provide a medical image synthesis apparatus and method that can increase tissue density information while further improving image resolution and contrast, thereby enabling clear display of lesions and improving lesion detection efficiency.

[0006] According to one aspect of the embodiments of this application, a medical image synthesis apparatus is provided, wherein the apparatus includes:

[0007] An acquisition unit is used to acquire a first medical image and a second medical image;

[0008] A registration unit is used to register the first medical image with the second medical image;

[0009] The determining unit is used to determine the first parameter value of each pixel position in the registered first medical image and the second parameter value of each pixel position in the second medical image;

[0010] The generation unit is used to multiply the first parameter value and the second parameter value at the same pixel position of the registered first medical image and the second medical image, and generate synthetic image data based on the multiplication result.

[0011] In some embodiments, the first medical image is an anatomical image, and the second medical image is a molecular image.

[0012] In some embodiments, the first parameter value is the corresponding value of the linear attenuation coefficient of a tissue or organ to radiation, and the second parameter value is the standard uptake value.

[0013] In some embodiments, the device further includes:

[0014] A preprocessing unit is used to perform at least one of the following preprocessing operations on the first medical image and the second medical image: resampling, image enhancement, and image denoising.

[0015] In some embodiments, the device further includes:

[0016] A correction unit is used to correct the first parameter value and / or the second parameter value;

[0017] Furthermore, the generation unit multiplies the first parameter value and the corrected second parameter value at the same pixel location in the registered first and second medical images; or,

[0018] Multiply the corrected first parameter value and second parameter value at the same pixel position in the registered first and second medical images; or

[0019] Multiply the corrected first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images.

[0020] In some embodiments, the generation unit uses the multiplication result as the pixel value at the corresponding pixel location in the synthesized image; or...

[0021] The generation unit determines the mapping value corresponding to the multiplication result as the pixel value at the same pixel position in the synthesized image.

[0022] In some embodiments, the image synthesized by the generating unit is a grayscale image or a color image.

[0023] According to one aspect of the embodiments of this application, a medical image synthesis method is provided, wherein the method includes:

[0024] Acquire the first medical image and the second medical image;

[0025] The first medical image and the second medical image are registered;

[0026] Determine the first parameter value of each pixel position in the registered first medical image and the second parameter value of each pixel position in the second medical image;

[0027] The first parameter value and the second parameter value at the same pixel position in the registered first medical image and the second medical image are multiplied together, and the composite image data is generated based on the multiplication result.

[0028] In some embodiments, the method further includes:

[0029] Correct the first parameter value and / or the second parameter value;

[0030] Furthermore, multiplying the first parameter value and the second parameter value at the same pixel position in the registered first medical image and second medical image includes:

[0031] Multiply the first parameter value at the same pixel location in the registered first and second medical images by the corrected second parameter value; or,

[0032] Multiply the corrected first parameter value and second parameter value at the same pixel position in the registered first and second medical images; or

[0033] Multiply the corrected first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images.

[0034] According to another aspect of the embodiments of this application, a medical image synthesis apparatus is provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the computer program to implement the medical image synthesis method as described above.

[0035] According to another aspect of the embodiments of this application, a storage medium storing a computer-readable program is provided, wherein the computer-readable program causes a computer to perform the medical image synthesis method as described above in a medical image synthesis apparatus.

[0036] One of the beneficial effects of this application embodiment is that: multiplying the first parameter value of the first medical image with the second parameter value of the corresponding pixel position of the second medical image to generate a composite image based on the multiplication result, the multiplication of the first parameter value (e.g., the CT value of a CT image) and the second parameter value (e.g., the SUV value of a PET image) is equivalent to amplifying the difference between the pixel values ​​of the normal tissue and the lesion tissue, making the composite image have higher contrast and resolution, thereby making the lesion area clearer and improving the lesion detection efficiency.

[0037] Referring to the following description and accompanying drawings, specific implementation methods of the embodiments of this application are disclosed in detail, indicating how the principles of the embodiments of this application can be adopted. It should be understood that the implementation methods of this application are not limited in scope. Within the spirit and scope of the appended claims, the implementation methods of this application include many changes, modifications, and equivalents. Attached Figure Description

[0038] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other implementation methods based on these drawings without creative effort. In the drawings:

[0039] Figure 1 This is a schematic diagram of a medical image synthesis method according to an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of matrix dot product according to an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of a grayscale composite image according to an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of a color composite image according to an embodiment of this application;

[0043] Figure 5A and Figure 5B These are schematic diagrams of the first and second medical images in embodiments of this application;

[0044] Figure 5C This is a schematic diagram of an image synthesized using existing methods;

[0045] Figure 6 This is a schematic diagram of a medical image synthesis device according to an embodiment of this application;

[0046] Figure 7 This is a schematic diagram of a medical image synthesis device according to an embodiment of this application. Detailed Implementation

[0047] Referring to the accompanying drawings, the foregoing and other features of the embodiments of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of the embodiments of this application can be adopted. It should be understood that this application is not limited to the described embodiments; rather, the embodiments of this application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0048] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or chronological order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0049] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0050] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. The term "comprising / including" as used herein means the presence of a feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components.

[0051] The devices described herein for acquiring medical imaging data are applicable to a variety of medical imaging modalities, including but not limited to computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, positron emission tomography (PET) devices, single photon emission computed tomography (SPECT) devices, PET / CT, PET / MR, or any other suitable medical imaging devices.

[0052] A system for acquiring medical images may include the aforementioned medical imaging equipment, a separate computer device connected to the medical imaging equipment, or a computer device connected to an internet cloud, which is connected via the internet to the medical imaging equipment or a storage device for storing medical images. The imaging method may be implemented independently or in combination by the aforementioned medical imaging equipment, the computer device connected to the medical imaging equipment, and the computer device connected to the internet cloud.

[0053] For example, CT scans use X-rays to scan a continuous cross-section of a part of the object being scanned. The detector receives the X-rays that pass through the layer and converts them into visible light, or directly converts the received photon signals and then processes them to reconstruct the image. MRI is based on the principle of nuclear magnetic resonance of atomic nuclei. It emits radio frequency pulses to the object being scanned and receives the electromagnetic signals emitted by the object, and then reconstructs an image.

[0054] PET uses a cyclotron to accelerate charged particles and bombard a target nucleus, producing positron-carrying radioactive nuclides through nuclear reactions. These nuclides are then synthesized into imaging agents, which are introduced into the body and positioned in the target organ. During their decay, these nuclides emit positively charged electrons. These positrons travel a short distance in the tissue before interacting with electrons in the surrounding material, resulting in annihilation radiation and the emission of two photons with opposite directions and equal energy. PET imaging uses a series of paired probes arranged at 180° intervals and connected to coincidence circuits to detect the photons of the annihilation radiation produced by the tracer outside the body. The acquired information is processed by a computer to obtain a reconstructed image.

[0055] SPECT uses radioactive isotopes as tracers. These tracers are injected into the human body, causing them to concentrate on the organ being tested, thus making the organ a source of gamma rays. Outside the body, a detector that rotates around the body records the distribution of radioactivity in the organ tissue. The detector obtains a set of data by rotating one angle, and several sets of data by rotating one revolution. Based on this data, a series of tomographic images can be created, and the computer reconstructs the image in a cross-sectional manner.

[0056] PET and SPECT, starting from the molecular level, extend histopathological examination to the visualization of local tissue biochemistry, providing images of human physiological metabolism. They excel in functional imaging and can detect functional and metabolic changes in the occurrence and development of diseases. CT and MRI, on the other hand, excel in accurately reflecting morphological and structural changes. In existing methods, CT or MRI can be used to attenuate PET or SPECT images, that is, to integrate PET or SPECT with CT or MRI, achieving complementarity between functional and anatomical image information, so as to achieve better identification and diagnosis.

[0057] Furthermore, medical imaging workstations can be located locally on the medical imaging equipment, meaning they are situated close to the equipment. The workstation and equipment can share a location within the scanning room, radiology department, or the same hospital. In contrast, the medical image cloud platform analysis system can be located away from the medical imaging equipment, for example, in the cloud where it communicates with the equipment.

[0058] As an example, after a medical institution completes an imaging scan using medical imaging equipment, the scanned data is stored in a storage device. A medical imaging workstation can directly read the scanned data and perform image processing through its processor. As another example, a medical image cloud platform analysis system can remotely access medical images stored in the storage device to provide "Software as a Service" (SaaS). SaaS can exist between hospitals, between hospitals and imaging centers, or between hospitals and third-party online medical service providers.

[0059] The embodiments of this application are described in detail below.

[0060] First aspect of the embodiments

[0061] This application provides a method for medical image synthesis. Figure 1 This is a schematic diagram of a medical image synthesis method according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0062] 101. Acquire the first medical image and the second medical image;

[0063] 102. Register the first medical image with the second medical image;

[0064] 103. Determine the first parameter value of each pixel position on the registered first medical image and the second parameter value of each pixel position on the second medical image;

[0065] 104. Multiply the first parameter value and the second parameter value at the same pixel position of the registered first medical image and the second medical image, and generate synthetic image data based on the multiplication result.

[0066] It is worth noting that the above appendix Figure 1 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted, and other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 1 The records.

[0067] In some embodiments, the first medical image and the second medical image are images corresponding to the same scanned object or the same scanned object region of interest. The first medical image and the second medical image are medical images obtained based on different types of imaging principles. For example, the first medical image is an anatomical image that can provide precise anatomical localization of tissues or organs, including CT images or MRI images, etc.; the second medical image is a molecular image that can provide information such as the function or metabolism of tissues or organs, including PET images or SPECT images, etc. The embodiments of this application are not intended to limit the scope of the invention, and examples will not be provided here.

[0068] In some embodiments, in step 101, first medical image and second medical image data are acquired to obtain image data of the first medical image and image data of the second medical image. The data type of the image data can be a DICOM file or a NIFTI file. The image data (or data file) may include the following information: pixel depth, photometric interpretation (indicating whether the image is displayed as a monochrome or color image; for example, the first medical image is usually a monochrome image, and the second medical image is usually a color image, but this embodiment does not limit this), metadata (including image matrix dimensions, spatial precision, etc.), and pixel data (the size of pixel values ​​at each location, represented using integer or floating-point data types). The specific format of the DICOM or NIFTI file can be found in existing technologies and will not be elaborated here. For example, the pixel data can be represented using a matrix, where the rows of the matrix correspond to the height of the image (in pixels), the columns of the matrix correspond to the width of the image (in pixels), and each element of the matrix corresponds to a pixel at a corresponding position in the image; the element value is the pixel value (grayscale value).

[0069] In some embodiments, in step 102, the first medical image and the second medical image are registered. For example, the second medical image can be matched to the first medical image using a rigid registration method based on mutual information. For instance, the mutual information between the first and second medical images is calculated, and the mutual information is continuously calculated after various translations and rotations of the second medical image. When the mutual information is maximized, the second medical image is matched to the first medical image. Simultaneously, the rotation matrix and translation vector for the second medical image registration are obtained. The second medical image is then processed using this rotation matrix and translation vector to achieve the registration of the first and second medical images. The above is merely an illustrative example, and the embodiments of this application are not intended to limit the scope. Other registration methods can also be used for registration. Furthermore, the first medical image can also be matched to the second medical image; these will not be listed here.

[0070] In some embodiments, in order to improve the image resolution and the image fusion result, the first medical image and the second medical image may be preprocessed before image fusion, that is, the method may further include: resampling the first medical image and the second medical image, image enhancement, and image denoising as at least one preprocessing.

[0071] For example, the first medical image and the registered second medical image are resampled (or the second medical image and the registered first medical image are resampled). This resampling can be based on the nearest neighbor interpolation method. According to the ratio of the width (or height) of the target image to the width (or height) of the source image (scaling ratio), the pixel value of the source image at the relative position is taken as the pixel value of the target image. For example, the scaling ratio in each direction can be set as: x-spacing = 1; y-spacing = 1; z-spacing = 2. The above is only an example illustration, but the embodiments of this application are not limited thereto. For example, linear interpolation and other methods can also be used. For details, please refer to the prior art, which will not be elaborated here.

[0072] For example, image enhancement can be performed on the first medical image and the registered second medical image (or on the second medical image and the registered first medical image). This image enhancement can use the Laplacian method, but this embodiment of the application does not limit it. For example, other methods such as histogram averaging can also be used. For details, please refer to the prior art, which will not be elaborated here.

[0073] For example, image denoising can be performed on the first medical image and the registered second medical image (or on the second medical image and the registered first medical image). This image denoising can use Gaussian blurring. For example, the Gaussian blurring parameters (Gaussian blur radius) of the first medical image and the second medical image can be set respectively. However, this embodiment of the application is not limited to this. For example, other methods such as median filtering can also be used. For details, please refer to the prior art. They will not be described in detail here.

[0074] It should be noted that this embodiment is not limited to the execution order of preprocessing steps such as resampling, image enhancement, and image denoising. Other preprocessing operations can also be added. Those skilled in the art can make appropriate modifications based on the above description, and this embodiment is not intended to be limiting.

[0075] In some embodiments, in step 103, a first parameter value for each pixel location on the registered first medical image and a second parameter value for each pixel location on the second medical image are determined. For example, when the first medical image is a CT image and the second medical image is a PET or SPECT, the first parameter value is the corresponding value of the linear attenuation coefficient of the tissue or organ to radiation, i.e., the CT value (unit: HU), which can measure the absorption rate of the tissue or organ to radiation. The second parameter value is the standard uptake value (SUV, unit: g / mL), which refers to the ratio of the specific activity measured in the region of interest to the dose administered per unit body weight, and can be used to evaluate the benign or malignant nature of lesions. The determination of the first and second parameter values ​​is explained below.

[0076] In some embodiments, the first parameter value can be determined based on the first pixel value (grayscale value) at each pixel position on the first medical image. For example, if the image data of the first medical image is stored in the form of a DICOM file, the first pixel value can be multiplied by the slope and the intercept to obtain the first parameter value. The slope and the intercept are also included in the tag of the DICOM file. When the weighting coefficient is 1 and the predetermined value is 0, the first pixel value is equal to the first parameter value. If the image data of the first medical image is stored in the form of an NIFTI file, the first pixel value can be converted into the first parameter value using a predetermined function. The predetermined function can be referred to in the prior art and will not be described in detail here.

[0077] As shown in Table 1 below, the first parameter value of water is usually 0, the first parameter value of air is usually -1000, the first parameter value of fat is usually less than 0, the first parameter value of the bladder is usually 0, the first parameter value of the intestine is usually 0 or -1000, and the first parameter value of solid organs can be set to A (A is not equal to 0). The value of A can be referred to the existing technology, and will not be listed here.

[0078] In some embodiments, the second parameter value can be determined based on the second pixel value at each pixel position on the registered second medical image. For example, if the image data of the second medical image is stored in the form of a DICOM file, the second pixel value can be multiplied by the slope and the intercept, and then the second parameter value can be calculated by combining the injection scan time and the half-life of the radionuclide (the calculation formula can refer to the prior art, and will not be exemplified here). The slope and the intercept are also included in the tag of the DICOM file. If the image data of the second medical image is stored in the form of an NIFTI file, in addition to calculating the second parameter value based on the second pixel value using the prior art, the second parameter value also needs to be corrected based on the scan object information (e.g., the bw coefficient) to determine the corrected second parameter value, and the corrected second parameter value is used as the second parameter value at each pixel position on the second medical image.

[0079] As shown in Table 1 below, the second parameter value of water is usually greater than 0, the second parameter value of air is usually 0, the second parameter value of fat is usually greater than 0 or 0, the second parameter value of bladder is usually greater than 0, the second parameter value of intestine is usually 0, and the second parameter value of solid organs can be set to B (B is not equal to 0). The value of B can be referred to the existing technology, and will not be listed one by one here.

[0080] In some embodiments, in 104, the first parameter value and the second parameter value at the same pixel position are multiplied. For example, after registration and preprocessing, the width and height of the first medical image and the second medical image are W and H respectively (W and H are positive integers), which means that there are W×H pixel positions. For example, the pixel positions can be numbered in a row-first-column order. The aforementioned same pixel position refers to the pixel position with the same number. For ease of description, the product of the first parameter value and the second parameter value at the same pixel position is called the third parameter value (the unit is, for example, HU·g / mL). As shown in Table 1 below, the third parameter value of water is 0, the third parameter value of air is 0, the third parameter value of fat is usually less than 0 or is 0, the third parameter value of the bladder is 0, the third parameter value of the intestine is usually less than 0 or is 0, and the third parameter value of solid organs can be set to A×B (A×B is not equal to 0). Examples will not be given here.

[0081] Generally, a higher degree of malignancy corresponds to a higher SUV (potentially negative lesion). Clinically, the SUV value is often used to differentiate between malignant tumors and lesions, and to indicate the degree of malignancy. Based on clinical experience, if the SUV is greater than 2.5, it should be considered a malignant tumor, with a range of 2 to 2.5 being the borderline. If it is less than 2.0, it can be considered a benign lesion. Therefore, for lesion areas, especially small lesions, lesions in blood vessel walls and hollow organs, multiplying the first parameter value and the second parameter value is equivalent to amplifying the pixel value at the corresponding location, resulting in higher contrast and resolution of the synthesized image, thus making the lesion area clearer and improving the lesion detection efficiency.

[0082] Furthermore, the meaning of the third parameter value differs from that of the second and first parameter values. Therefore, the synthesized image differs in nature from the original image, reflecting tissue density information in addition to basic anatomical and metabolic information.

[0083] Table 1

[0084] name First parameter value Second parameter value Third parameter value water 0 ＞0 0 Air -1000 0 0 Fat ＜0 0 or > 0 0 or < 0 bladder 0 ＞0 0 Intestines 0 or -1000 0 0 solid organs A B A×B

[0085] In some embodiments, the first parameter value at each pixel position of the first medical image can be represented by a first image matrix, and the second parameter value at each pixel position of the second medical image can be represented by a second image matrix. In step 104, multiplying the first parameter value and the second parameter value at the same pixel position is equivalent to multiplying the first image matrix and the second image matrix by matrix dot product. Figure 2 This is an example diagram of matrix dot product, such as... Figure 2 As shown, the first and second image matrices are both W×H dimensional matrices, and the result of the dot product, the third image matrix, is also a W×H dimensional matrix.

[0086] In some embodiments, the first parameter value and the second parameter value can be directly multiplied, or a weighting coefficient can be used to correct the first parameter value and / or the second parameter value before multiplication. Examples are given below.

[0087] For example, the first parameter value is corrected using a first weighting coefficient (the first weighting coefficient is multiplied by the first parameter value), the second parameter value is corrected using a second weighting coefficient (the second weighting coefficient is multiplied by the second parameter value), the corrected first parameter value and the corrected second parameter value at the same pixel position are multiplied together, or the first weighting coefficient is multiplied by the first image matrix, then the second weighting coefficient is multiplied by the second image matrix, and then the corrected first image matrix and the corrected second image matrix are multiplied by a dot product.

[0088] For example, the first parameter value is corrected using a first weighting coefficient (the first weighting coefficient is multiplied by the first parameter value), the second parameter value is corrected using a second weighting coefficient (the second weighting coefficient is multiplied by the second parameter value), the corrected first parameter value and the corrected second parameter value at the same pixel position are multiplied together, or the first weighting coefficient is multiplied by the first image matrix, then the second weighting coefficient is multiplied by the second image matrix, and then the corrected first image matrix and the corrected second image matrix are multiplied by a dot product.

[0089] For example, the first parameter value can be corrected using only the first weighting coefficient (the first weighting coefficient is multiplied by the first parameter value), the corrected first parameter value and the second parameter value at the same pixel position can be multiplied together, or the first weighting coefficient can be multiplied by the first image matrix and then multiplied by the second image matrix.

[0090] For example, the second parameter value can be corrected using only the second weighting coefficient (the second weighting coefficient is multiplied by the second parameter value), the corrected second parameter value at the same pixel position can be multiplied by the first parameter value, or the second weighting coefficient can be multiplied by the second image matrix and then multiplied by the first image matrix.

[0091] The first and second weighting coefficients mentioned above can be determined as needed. Since the above weighting coefficients are used for correction, the corresponding pixel values ​​can be further amplified after multiplication, resulting in higher contrast and resolution of the synthesized image, thus making the lesion area clearer.

[0092] In some embodiments, the multiplication result can be used as the pixel value at the same pixel position in the synthesized image; or, the mapping value corresponding to the multiplication result can be determined as the pixel value at the same pixel position in the synthesized image.

[0093] For example, the third parameter value can be directly used as the pixel grayscale value to generate composite image data, which is a grayscale image. Figure 3This is a schematic diagram of the grayscale composite image; alternatively, the third parameter value can be used as the pixel grayscale value and then converted to RGB values ​​(mapped values) to generate composite image data, which is a color image. Figure 4 This is a schematic diagram of the color composite image. The method for converting grayscale values ​​to RGB values ​​can refer to existing technologies, which will not be elaborated here.

[0094] Figure 5A This is a schematic diagram of the first medical image. Figure 5B This is a schematic diagram of the second medical image. Figure 5C This is a schematic diagram of a conventional PET / CT fusion image, such as... Figure 3 and Figure 4 As shown, with Figure 5A , Figure 5B and Figure 5C After comparison, the synthesized image has higher contrast and resolution, as shown by the arrow in the attached figure, and the lesion area is clearer.

[0095] For example, the third image matrix can be converted into synthetic image data and stored in a predetermined data file type. The predetermined data file type can be an existing DICOM file or NIFTI file, or it can be a newly defined file type. This application embodiment does not limit it in this way.

[0096] In some embodiments, when the first medical image is a CT image and the second medical image is a PET or SPECT image, the composite image is represented as SyPCT or SySPECT. This composite image can be used for tissue density analysis in conjunction with quantitative analysis of functional metabolism, enzyme or gene expression, as well as radiomics analysis.

[0097] It should be noted that the medical images described in this application embodiment are suitable for any region of interest of any scanned object, and this application embodiment is not intended to limit them.

[0098] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0099] As can be seen from the above embodiments, multiplying the first parameter value of the first medical image with the second parameter value of the corresponding pixel position of the second medical image to generate a composite image based on the multiplication result, the multiplication of the first parameter value and the second parameter value is equivalent to amplifying the difference between the pixel values ​​of the normal tissue and the lesion tissue, so that the composite image has higher contrast and resolution, thereby making the lesion area clearer and improving the lesion detection efficiency.

[0100] Second aspect of the embodiments

[0101] This application provides a medical image synthesis device, and the contents that are the same as those in the first aspect of the embodiment will not be repeated.

[0102] Figure 6 This is a schematic diagram of a medical image synthesis apparatus according to an embodiment of this application. Figure 6 As shown, the medical image synthesis device 600 includes:

[0103] Acquisition unit 601 is used to acquire a first medical image and a second medical image;

[0104] Registration unit 602 is used to register the first medical image with the second medical image;

[0105] The determining unit 603 is used to determine the first parameter value of each pixel position on the registered first medical image and the second parameter value of each pixel position on the second medical image.

[0106] The generation unit 604 is used to multiply the first parameter value and the second parameter value at the same pixel position of the registered first medical image and the second medical image, and generate synthetic image data based on the multiplication result.

[0107] In some embodiments, the implementation of the acquisition unit 601, registration unit 602, determination unit 603, and generation unit 604 can refer to 101-104 of the first aspect embodiment, and repeated details will not be described again.

[0108] In some embodiments, the first medical image is an anatomical image, and the second medical image is a molecular image.

[0109] In some embodiments, the first parameter value is the corresponding value of the linear attenuation coefficient of a tissue or organ to radiation, and the second parameter value is the standard uptake value.

[0110] In some embodiments, the device further includes (not shown):

[0111] A preprocessing unit is used to perform at least one of the following preprocessing operations on the first medical image and the second medical image: resampling, image enhancement, and image denoising.

[0112] In some embodiments, the device further includes (not shown):

[0113] A correction unit is used to correct the first parameter value and / or the second parameter value;

[0114] Furthermore, the generation unit 604 multiplies the first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images; or,

[0115] Multiply the corrected first parameter value and second parameter value at the same pixel position in the registered first and second medical images; or

[0116] Multiply the corrected first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images.

[0117] In some embodiments, the generation unit 604 uses the multiplication result as the pixel value at the corresponding pixel location in the synthesized image; or...

[0118] The generation unit 604 determines the mapping value corresponding to the multiplication result as the pixel value at the same pixel position in the synthesized image.

[0119] In some embodiments, the synthesized image is a grayscale image or a color image.

[0120] For the sake of simplicity, Figure 6 The diagram only exemplifies the connection relationships or signal flow between various components or modules; however, those skilled in the art should understand that various related technologies, such as bus connections, can be employed. The aforementioned components or modules can be implemented using hardware facilities such as processors and memory; this application does not limit the scope of the embodiments.

[0121] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0122] As can be seen from the above embodiments, multiplying the first parameter value of the first medical image with the second parameter value of the corresponding pixel position of the second medical image to generate a composite image based on the multiplication result, the multiplication of the first parameter value and the second parameter value is equivalent to amplifying the difference between the pixel values ​​of the normal tissue and the lesion tissue, so that the composite image has higher contrast and resolution, thereby making the lesion area clearer and improving the lesion detection efficiency.

[0123] Third aspect of the embodiments

[0124] This application provides a medical image synthesis device, including the medical image synthesis apparatus 600 as described in the second aspect of the embodiment, the contents of which are incorporated herein by reference. This medical image synthesis device may be, for example, a computer, server, workstation, laptop computer, smartphone, etc.; however, this application is not limited thereto.

[0125] Figure 7 This is a schematic diagram of a medical image synthesis device according to an embodiment of this application. Figure 7 As shown, the medical image synthesis device 700 may include one or more processors (e.g., a central processing unit, CPU) 710 and one or more memories 720; the memories 720 are coupled to the processors 710. The memories 720 may store various data and training models, etc.; in addition, they also store information processing programs 721, and execute the programs 721 under the control of the processors 710.

[0126] In some embodiments, the functionality of the medical image synthesis apparatus 600 is integrated into a processor 710. The processor 710 is configured to implement the medical image synthesis method as described in the embodiments of the first aspect.

[0127] In some embodiments, the medical image synthesis device 600 is configured separately from the processor 710. For example, the medical image synthesis device 600 can be configured as a chip connected to the processor 710, and the functions of the medical image synthesis device 600 can be realized through the control of the processor 710.

[0128] For example, the processor 710 is configured to perform the following control: acquire a first medical image and a second medical image; register the first medical image and the second medical image; determine a first parameter value for each pixel position in the registered first medical image and a second parameter value for each pixel position in the second medical image; multiply the first parameter value and the second parameter value at the same pixel position in the registered first medical image and the second medical image, and generate composite image data based on the multiplication result.

[0129] For example, the processor 710 is configured to perform at least one preprocessing operation on the first medical image and the second medical image, namely, resampling, image enhancement, and image denoising.

[0130] For example, the processor 710 is configured to perform the following control: correct the first parameter value and / or the second parameter value; multiply the first parameter value and the corrected second parameter value at the same pixel position of the registered first medical image and the second medical image; or multiply the corrected first parameter value and the corrected second parameter value at the same pixel position of the registered first medical image and the second medical image; or multiply the corrected first parameter value and the corrected second parameter value at the same pixel position of the registered first medical image and the second medical image.

[0131] For example, the processor 710 is configured to perform the following control: using the multiplication result as the pixel value of the synthesized image corresponding to the same pixel position; or, determining the mapping value corresponding to the multiplication result as the pixel value of the synthesized image corresponding to the same pixel position.

[0132] In some embodiments, the processor 710 may be implemented with reference to the embodiments of the first aspect, which will not be repeated here.

[0133] In addition, such as Figure 7 As shown, the medical image synthesis apparatus 700 may further include: an input / output (I / O) device 730 and a display 740 (displaying a first medical image, a second medical image, and a synthesized image), etc.; the functions of the above components are similar to those in the prior art, and will not be described in detail here. It is worth noting that the medical image synthesis apparatus 700 is not necessarily required to include... Figure 7 All components shown; in addition, the medical image synthesis device 700 may also include Figure 7 For components not shown, please refer to relevant technologies.

[0134] As can be seen from the above embodiments, multiplying the first parameter value of the first medical image with the second parameter value of the corresponding pixel position of the second medical image to generate a composite image based on the multiplication result, the multiplication of the first parameter value and the second parameter value is equivalent to amplifying the difference between the pixel values ​​of the normal tissue and the lesion tissue, so that the composite image has higher contrast and resolution, thereby making the lesion area clearer and improving the lesion detection efficiency.

[0135] This application also provides a computer-readable program, wherein when executed in a medical image synthesis apparatus, the program causes a computer in the medical image synthesis apparatus to perform the medical image synthesis method as described in the first aspect of the embodiment.

[0136] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to perform the medical image synthesis method as described in the first aspect of the embodiment in a medical image synthesis apparatus.

[0137] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

[0138] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0139] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0140] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0141] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles thereof, and these modifications and variations are also within the scope of the present application.

Claims

1. A medical image synthesis device, characterized in that, The device includes: An acquisition unit is used to acquire a first medical image and a second medical image, wherein the first medical image and the second medical image are medical images obtained based on different types of imaging principles; A registration unit is used to register the first medical image and the second medical image; The determining unit is used to determine the first parameter value of each pixel position in the registered first medical image and the second parameter value of each pixel position in the second medical image; A correction unit is configured to apply a first weighting coefficient to the first parameter value and a second weighting coefficient to the second parameter value to correct the first parameter value and / or the second parameter value. The generation unit is used to multiply the first parameter value and the second parameter value at the same pixel position of the registered first medical image and the second medical image pixel by pixel, and generate synthetic image data based on the multiplication result; The step of multiplying the first parameter value and the second parameter value at the same pixel position in the registered first medical image and the second medical image pixel by pixel includes: Multiply the first parameter value at the same pixel location in the registered first and second medical images by the corrected second parameter value; or, Multiply the corrected first parameter value and second parameter value at the same pixel position in the registered first and second medical images; or Multiply the corrected first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images.

2. The apparatus according to claim 1, characterized in that, The first medical image is an anatomical image, and the second medical image is a molecular image.

3. The apparatus according to claim 1 or 2, characterized in that, The first parameter value is the corresponding value of the linear attenuation coefficient at each pixel position on the first medical image, where the linear attenuation coefficient is the linear attenuation coefficient of a tissue or organ to radiation, and the second parameter value is the standard uptake value at each pixel position on the second medical image.

4. The apparatus according to claim 1, characterized in that, The device further includes: A preprocessing unit is used to perform at least one of the following preprocessing operations on the first medical image and the second medical image: resampling, image enhancement, and image denoising.

5. The apparatus according to claim 1, characterized in that, The generation unit uses the multiplication result as the pixel value at the corresponding pixel position in the synthesized image; or... The generation unit determines the mapping value corresponding to the multiplication result as the pixel value at the same pixel position in the synthesized image.

6. The apparatus according to claim 1, characterized in that, The image synthesized by the generation unit is a grayscale image or a color image.

7. A method for synthesizing medical images, characterized in that, The method includes: Acquire a first medical image and a second medical image, wherein the first medical image and the second medical image are medical images obtained based on different types of imaging principles; The first medical image and the second medical image are registered; Determine the first parameter value of each pixel position in the registered first medical image and the second parameter value of each pixel position in the second medical image; A first weighting coefficient is applied to the first parameter value, and a second weighting coefficient is applied to the second parameter value to correct the first parameter value and / or the second parameter value. The first parameter value and the second parameter value at the same pixel position in the registered first medical image and the second medical image are multiplied pixel by pixel, and the composite image data is generated based on the multiplication result. The step of multiplying the first parameter value and the second parameter value at the same pixel position in the registered first medical image and the second medical image pixel by pixel includes: Multiply the first parameter value at the same pixel location in the registered first and second medical images by the corrected second parameter value; or, Multiply the corrected first parameter value and second parameter value at the same pixel position in the registered first and second medical images; or Multiply the corrected first parameter value and the corrected second parameter value at the same pixel position in the registered first and second medical images.

8. A storage medium storing a computer-readable program, characterized in that, The computer-readable program causes a computer to perform the medical image synthesis method of claim 7 in a medical image synthesis device.

Images