Image reconstruction method and apparatus based on deep learning

Abstract

The application relates to a deep learning-based image reconstruction method and device. The method comprises the following steps: acquiring image data of different scanning modes of a user; performing image registration processing on the image data of each scanning mode to obtain registration image data of the user, and performing image processing and image optimization processing on the registration image data to obtain standardized image data of the user; and based on the standardized image data, constructing target image data of the user through an image reconstruction model. The method can improve the imaging accuracy of early diagnosis images of Parkinson's disease.

Description

Technical Field

[0001] This application relates to the fields of image reconstruction and medical imaging technology, and in particular to an image reconstruction method and apparatus based on deep learning. Background Technology

[0002] Medical imaging is a crucial tool for disease diagnosis. CT (Computed Tomography) and PET (Positron Emission Tomography) are two commonly used imaging techniques in the diagnosis of Parkinson's disease. CT generates clear images of anatomical structures through X-ray scanning and is widely used in the diagnosis of bone and soft tissue lesions, but it cannot reflect the metabolic activity of tissues. PET generates images reflecting metabolic and functional information through tracer injection, especially 18F-FDG PET, which can quantitatively detect brain glucose metabolism. Abnormal changes in 18F-FDG PET often precede abnormalities seen in anatomical imaging. Therefore, improving the accuracy of image acquisition and the comprehensiveness of detailed structural feedback are current research priorities in medical imaging technology.

[0003] Traditional technical solutions employ deep learning methods for image reconstruction and fusion. Traditional multimodal image fusion relies heavily on data registration and lacks precise control over the consistency between functional information and anatomical structures. This makes it difficult to accurately detect early lesions of Parkinson's disease, especially subtle structural changes, and also provides limited support for quantitative analysis of PET images, resulting in poor imaging accuracy for early diagnosis of Parkinson's disease. Summary of the Invention

[0004] Therefore, it is necessary to provide a deep learning-based image reconstruction method, apparatus, computer device, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0005] Firstly, this application provides a deep learning-based image reconstruction method, including:

[0006] Acquire image data for users using different scanning methods;

[0007] Image registration processing is performed on the image data of each of the aforementioned scanning methods to obtain the registered image data of the user, and the registered image data is further processed to perform image optimization processing to obtain the standardized image data of the user.

[0008] Based on the standardized image data, the user's target image data is constructed using an image reconstruction model.

[0009] Optionally, the step of performing image registration processing on the image data of each of the scanning methods to obtain the registered image data of the user includes:

[0010] Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method;

[0011] Based on the regional structure data corresponding to each of the scanning methods, the image data of each of the scanning methods are processed by an image displacement registration program to obtain the sub-registered images corresponding to each of the scanning methods.

[0012] The sub-registration images corresponding to all scanning methods are used as the user's registration image data.

[0013] Optionally, the scanning method includes fluoroscopic scanning and glucose metabolism scanning. The step of performing image optimization processing on the registered image data to obtain the user's standardized image data includes:

[0014] The grayscale value of the sub-registration image corresponding to the glucose metabolism scanning mode is truncated to obtain the grayscale value truncated image corresponding to the glucose metabolism scanning mode.

[0015] The first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image are identified respectively. Based on the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, a pixel normalization processing strategy is used to obtain the fluoroscopic scan image corresponding to the fluoroscopic scanning method and the metabolic scan image corresponding to the grayscale truncated image.

[0016] Image data of the target layer range in the fluoroscopic scan is extracted and used as the standardized fluoroscopic scan corresponding to the fluoroscopic scan. Image data of the target layer range in the metabolic scan is also extracted and used as the standardized metabolic scan corresponding to the metabolic scan.

[0017] The standardized fluoroscopic scan and the standardized metabolic scan are used as the user's standardized image data.

[0018] Optionally, the step of constructing the user's target image data based on the standardized image data using an image reconstruction model includes:

[0019] In response to the information upload operation of the staff, the system obtains the random noise information of the standardized fluoroscopic scan image and the prompt word information of the standardized fluoroscopic scan image.

[0020] The standardized perspective scan image, the random noise information of the standardized perspective scan image, the prompt word information of the standardized perspective scan image, and the standardized metabolic scan image are input into the image reconstruction model to generate the user's target image data.

[0021] Optionally, after constructing the user's target image data based on the standardized image data using an image reconstruction model, the method further includes:

[0022] Extract the target anomaly region from the target image data, and extract the anomaly region feature data of the target anomaly region;

[0023] Based on the abnormal region feature data of the target abnormal region, the abnormal type and abnormal information of the target abnormal region are identified through an image feature analysis model.

[0024] Based on the anomaly type and anomaly information of the target anomaly region, a regional anomaly guidance report is generated for the user.

[0025] Optionally, generating the user's regional anomaly guidance report based on the anomaly type and anomaly information of the target anomaly region includes:

[0026] Based on the anomaly type of the target anomaly region, identify the user's regional anomaly state and the degree of regional anomaly.

[0027] Based on the anomaly information of the target anomaly region, anomaly marking processing is performed on the user's target image data to obtain the user's image marking information. Based on the user's regional anomaly status, the user's regional anomaly degree, and the user's image marking information, a regional anomaly guidance report for the user is generated through a report guidance template.

[0028] Secondly, this application also provides an image reconstruction apparatus based on deep learning, comprising:

[0029] The acquisition module is used to acquire image data for different scanning methods used by the user;

[0030] The processing module is used to perform image registration processing on the image data of each of the scanning methods to obtain the registered image data of the user, and to perform image optimization processing on the registered image data to obtain the standardized image data of the user.

[0031] The construction module is used to construct the user's target image data based on the standardized image data and through an image reconstruction model.

[0032] Optionally, the processing module is specifically used for:

[0033] Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method;

[0034] Based on the regional structure data corresponding to each of the scanning methods, the image data of each of the scanning methods are processed by an image displacement registration program to obtain the sub-registered images corresponding to each of the scanning methods.

[0035] The sub-registration images corresponding to all scanning methods are used as the user's registration image data.

[0036] Optionally, the processing module is specifically used for:

[0037] The grayscale value of the sub-registration image corresponding to the glucose metabolism scanning mode is truncated to obtain the grayscale value truncated image corresponding to the glucose metabolism scanning mode.

[0038] The first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image are identified respectively. Based on the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, a pixel normalization processing strategy is used to obtain the fluoroscopic scan image corresponding to the fluoroscopic scanning method and the metabolic scan image corresponding to the grayscale truncated image.

[0039] Image data of the target layer range in the fluoroscopic scan is extracted and used as the standardized fluoroscopic scan corresponding to the fluoroscopic scan. Image data of the target layer range in the metabolic scan is also extracted and used as the standardized metabolic scan corresponding to the metabolic scan.

[0040] The standardized fluoroscopic scan and the standardized metabolic scan are used as the user's standardized image data.

[0041] Optionally, the building module is specifically used for:

[0042] In response to the information upload operation of the staff, the system obtains the random noise information of the standardized fluoroscopic scan image and the prompt word information of the standardized fluoroscopic scan image.

[0043] The standardized perspective scan image, the random noise information of the standardized perspective scan image, the prompt word information of the standardized perspective scan image, and the standardized metabolic scan image are input into the image reconstruction model to generate the user's target image data.

[0044] Optionally, the device further includes:

[0045] The extraction module is used to extract the target abnormal region and the abnormal region feature data of the target abnormal region from the target image data;

[0046] The identification module is used to identify the anomaly type and anomaly information of the target anomaly region based on the anomaly region feature data of the target anomaly region through an image feature analysis model.

[0047] The generation module is used to generate a regional anomaly guidance report for the user based on the anomaly type of the target anomaly region and the anomaly information of the target anomaly region.

[0048] Optionally, the generation module is specifically used for:

[0049] Based on the anomaly type of the target anomaly region, identify the user's regional anomaly state and the degree of regional anomaly.

[0050] Based on the anomaly information of the target anomaly region, anomaly marking processing is performed on the user's target image data to obtain the user's image marking information. Based on the user's regional anomaly status, the user's regional anomaly degree, and the user's image marking information, a regional anomaly guidance report for the user is generated through a report guidance template.

[0051] Thirdly, this application provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first aspects.

[0052] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0053] Fifthly, this application provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0054] The aforementioned deep learning-based image reconstruction method and apparatus acquire image data from different scanning methods of a user; perform image registration processing on the image data from each scanning method to obtain registered image data for the user; perform image optimization processing on the registered image data to obtain standardized image data for the user; and construct the user's target image data based on the standardized image data using an image reconstruction model. This scheme combines CT and PET images, performs registration and image optimization processing on the two images, thereby ensuring a balance between computational efficiency and image detail preservation, and accurately acquiring image data corresponding to effective regions in different scanning methods. Then, image reconstruction processing is performed, resulting in target image data that includes both precise structural information and image data reflecting structural features, as well as standardized image data from images acquired by different scanning methods. This allows the generated target image to comprehensively and holistically reflect various scanning information from the user's scans. In the process of generating images for early Parkinson's disease diagnosis, this target image can comprehensively and accurately reflect the subtle structural changes and lesion conditions of Parkinson's disease, thereby effectively improving the imaging accuracy of early Parkinson's disease diagnostic images. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating a deep learning-based image reconstruction method in one embodiment.

[0057] Figure 2 This is a flowchart illustrating an example of deep learning-based image reconstruction in one embodiment.

[0058] Figure 3 This is a structural block diagram of a deep learning-based image reconstruction device in one embodiment;

[0059] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] The deep learning-based image reconstruction method provided in this application can be applied to a deep learning-based intelligent control system for image reconstruction. This system can be applied to a terminal, which can be, but is not limited to, various personal computers, laptops, mid-range computers, etc. The terminal combines CT and PET images, performs registration and image optimization processing on the two images, thereby ensuring a balance between computational efficiency and image detail preservation. It can also accurately acquire image data corresponding to effective regions in different scanning methods. Then, image reconstruction processing is performed, resulting in target image data that includes both precise structural information and image data reflecting structural features, as well as standardized image data acquired from different scanning methods. This allows the generated target image to comprehensively and holistically reflect various scanning information from the user's scan. When this target image is used in the generation of images for early Parkinson's disease diagnosis, it can comprehensively and accurately reflect the subtle structural changes and lesion conditions of Parkinson's disease, thereby effectively improving the imaging accuracy of early Parkinson's disease diagnostic images.

[0062] In one exemplary embodiment, such as Figure 1 As shown, a deep learning-based image reconstruction method is provided. Taking the application of this method to a terminal as an example, the method includes the following steps S101 to S103. Wherein:

[0063] Step S101: Obtain image data for different scanning methods of the user.

[0064] In this embodiment, the terminal performs region scanning on the target area for generating disease images of the user using different scanning methods, acquiring image data of the user using different scanning methods. The target area for generating disease images includes, but is not limited to, brain regions for detecting neurodegenerative diseases such as Parkinson's disease and Alzheimer's disease, heart regions for detecting cardiovascular diseases, and other tumor-bearing areas for detecting tumors and other diseases. The scanning methods include, but are not limited to, CT scans and PET scans.

[0065] Step S102: Perform image registration processing on the image data of each scanning method to obtain the user's registered image data, and perform image optimization processing on the registered image data to obtain the user's standardized image data.

[0066] In this embodiment, the terminal performs image registration processing on the image data from each scanning method to obtain the user's registered image data. Then, it performs image optimization processing on the registered image data to obtain the user's standardized image data. The image registration processing involves shifting and registering the scanned images corresponding to different scanning methods. The image optimization methods include grayscale truncation, pixel normalization, and slice cropping. The specific registration and image optimization processes will be described in detail later.

[0067] Step S103: Based on standardized image data, construct the user's target image data through an image reconstruction model.

[0068] In this embodiment, the terminal constructs the user's target image data based on standardized image data and an image reconstruction model. This image reconstruction model is the Control-Net Stable Diffusion model designed in this solution. This model is based on the Stable Diffusion model and adds a Control-Net branch to achieve precise control. The specific construction process will be explained in detail later.

[0069] Based on the above scheme, by combining CT and PET images and performing registration and image optimization on the two images, it is possible to ensure a balance between computational efficiency and image detail preservation, while also accurately acquiring image data corresponding to the effective regions in different scanning methods. Then, image reconstruction processing is performed so that the generated target image data contains image data with precise structural information and reflective structural images, as well as standardized image data from images acquired by different scanning methods. This allows the generated target image to comprehensively and holistically reflect various scanning information from the user's scan. In particular, when this target image is used to generate images for early diagnosis of Parkinson's disease, it can comprehensively and accurately reflect the subtle structural changes and lesion conditions of Parkinson's disease, thereby effectively improving the imaging accuracy of images for early diagnosis of Parkinson's disease.

[0070] Optionally, image registration processing is performed on the image data of each scanning method to obtain the user's registered image data, including: identifying the region structure data corresponding to each scanning method based on the image data of each scanning method; performing image registration processing on the image data of each scanning method based on the region structure data corresponding to each scanning method through an image displacement registration program to obtain the sub-registered image corresponding to each scanning method; and using the sub-registered images corresponding to all scanning methods as the user's registered image data.

[0071] In this embodiment, the terminal identifies the regional structural data corresponding to each scanning method based on the image data from each scanning method. This regional structural data refers to the image structural data of the target region corresponding to the image from each scanning method, such as brain limb structure data or heart limb contour structure data.

[0072] Then, based on the regional structure data corresponding to each scanning method, the terminal performs image registration processing on the image data of each scanning method through an image displacement registration program to obtain sub-registered images corresponding to each scanning method. This image displacement registration program can be the registration program corresponding to the Elastix software.

[0073] Finally, the terminal uses the sub-registration images corresponding to all scanning methods as the user's registration image data.

[0074] Based on the above scheme, structural displacement registration is performed on image data from different scanning methods to adjust for displacement deviations caused by scanning errors in the imaging equipment. Registration ensures precise alignment of the anatomical structures in the two modalities, improving the accuracy of image registration and fusion.

[0075] Optionally, the scanning methods include fluoroscopic scanning and glucose metabolism scanning. The registered image data is processed and optimized to obtain standardized image data for the user. This includes: truncating the grayscale values ​​of the sub-registered images corresponding to the glucose metabolism scanning method to obtain a grayscale truncated image; identifying the first image pixel data corresponding to the sub-registered images of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, and based on these data, using a pixel normalization strategy, obtaining a fluoroscopic scan image and a metabolic scan image corresponding to the grayscale truncated image; extracting the image data of the target layer range from the fluoroscopic scan image as the standardized fluoroscopic scan image, and extracting the image data of the target layer range from the metabolic scan image as the standardized metabolic scan image; and using the standardized fluoroscopic scan image and the standardized metabolic scan image as the user's standardized image data.

[0076] In this embodiment, the terminal performs grayscale truncation processing on the sub-registered images corresponding to the glucose metabolism scanning mode to obtain a grayscale truncated image corresponding to the glucose metabolism scanning mode. Specifically, the terminal truncates the grayscale values ​​of the PET images, limiting the intensity values ​​to a reasonable range (e.g., within 12000), and performs normalization processing to adapt the data distribution to the model requirements. The result output is a processed grayscale histogram for quality verification.

[0077] Then, the terminal identifies the first image pixel data corresponding to the sub-registered image in the fluoroscopic scan mode, and the second image pixel data corresponding to the grayscale truncated image. The identification method for the image pixel data of each image is to use a pixel data identification program pre-installed on the terminal to identify the pixel values ​​of each image, obtain the pixel value distribution information of each image, and use this pixel value distribution information as the image pixel data for each image. This pixel data identification program is the program designed for image pixel identification in this solution. Specifically, the terminal adjusts the voxel size of the CT and PET images to unify the spatial resolution, while compressing the image size to 256×256 pixels to balance computational efficiency and image detail preservation.

[0078] Then, based on the first image pixel data corresponding to the sub-registered image of the perspective scanning method and the second image pixel data corresponding to the grayscale truncated image, the terminal obtains the perspective scan image corresponding to the perspective scanning method and the metabolic scan image corresponding to the grayscale truncated image through a pixel normalization processing strategy.

[0079] The terminal extracts image data of the target layer range from the fluoroscopic scan image as the corresponding standardized fluoroscopic scan image, and also extracts image data of the target layer range from the metabolic scan image as the corresponding standardized metabolic scan image. Specifically, the extracted target layer consists of the first 10 and last 10 irrelevant slices removed, retaining only the image data corresponding to the effective area in the middle.

[0080] Finally, the terminal will use the standardized fluoroscopic scan and the standardized metabolic scan as the user's standardized image data.

[0081] Based on the above scheme, by optimizing the image processing, not only is the representation degree of image feature data improved, but the image differences between images corresponding to different scanning methods can also be effectively reduced, thereby improving the image fusion efficiency and fusion accuracy.

[0082] Optionally, based on standardized image data, the user's target image data is constructed through an image reconstruction model, including: in response to the information upload operation of the staff, obtaining random noise information of the standardized fluoroscopic scan image and prompt word information of the standardized fluoroscopic scan image; inputting the standardized fluoroscopic scan image, the random noise information of the standardized fluoroscopic scan image, the prompt word information of the standardized fluoroscopic scan image, and the standardized metabolic scan image into the image reconstruction model to generate the user's target image data.

[0083] In this embodiment, in response to the information upload operation by the staff, the terminal obtains random noise information and prompt word information of the standardized perspective scan image. Specifically, the obtained random noise information is the image data corresponding to the noise pixels in the standardized perspective scan image, while the prompt word information includes the image data of key point pixels in the standardized perspective scan image, or the edge image data in the standardized perspective scan image, etc.

[0084] Finally, the terminal inputs the standardized fluoroscopic scan image, random noise information from the standardized fluoroscopic scan image, prompt word information from the standardized fluoroscopic scan image, and standardized metabolic scan image into the image reconstruction model to generate the user's target image data. Specifically, the processed CT image, random noise, and prompt words (such as key points or edge maps) are used as input into the Control-Net branch.

[0085] Then, the terminal control Control-Net processes the input data, generates conditional information, and passes it to the StableDiffusion master control model. Specifically, the trainable part learns a deep representation of the control variables to optimize the generation effect. The non-trainable part retains the original generation capability of StableDiffusion to ensure model stability.

[0086] Then, the terminal controls Stable Diffusion to combine prompts and Control-Net conditional outputs to generate PET images that are consistent with the functional information and anatomical structures of the CT images.

[0087] Based on the above scheme, the Control-Net-based Stable Diffusion image reconstruction model constructed in this scheme generates PET images that are consistent with the functional information and anatomical structure of CT images. This ensures that the fused image contains both structural data consistent with the functional information and anatomical structure of CT images, as well as features of PET images that reflect metabolic and functional information, thereby improving the comprehensiveness of the fused image's feature representation and the accuracy of displaying image details.

[0088] Optionally, after constructing the user's target image data based on standardized image data and an image reconstruction model, the method further includes: extracting the target abnormal region from the target image data and the abnormal region feature data of the target abnormal region; identifying the abnormal type of the target abnormal region and the abnormal information of the target abnormal region based on the abnormal region feature data of the target abnormal region and an image feature analysis model; and generating a regional abnormality guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region.

[0089] In this embodiment, the terminal extracts the target abnormal region and the abnormal region feature data of the target image data. The abnormal region feature data is image feature data extracted from the target image data of the target region using a feature extraction network based on a deep learning algorithm. This image feature data may include information on changes in brain region metabolic activity.

[0090] Then, based on the abnormal region feature data of the target abnormal region, the terminal uses an image feature analysis model to identify the abnormal type and abnormal information of the target abnormal region. This image feature analysis model is a convolutional neural network based on a deep learning algorithm. The extracted abnormal types of the target abnormal region include, for example, metabolic abnormalities, numerical abnormalities in brain regions, and structural abnormalities in brain regions. The abnormal information of the target abnormal region includes the range of metabolic values, the range of numerical values ​​in brain regions, and the range of structural data in brain regions.

[0091] Finally, based on the anomaly type and anomaly information of the target anomaly area, the terminal generates a regional anomaly guidance report for the user. This regional anomaly guidance report is used to guide staff on the scope, status, and severity of the anomaly in the user's target area. The specific generation process will be explained in detail later.

[0092] Based on the above scheme, by combining the reconstructed images with quantitative analysis using deep learning technology, medical staff can be guided to identify lesion areas earlier and more accurately, especially in the early stages of Parkinson's disease, thereby improving the accuracy and efficiency of guidance for medical staff.

[0093] Optionally, based on the anomaly type and anomaly information of the target anomaly region, a regional anomaly guidance report for the user is generated, including: identifying the user's regional anomaly status and the user's regional anomaly degree based on the anomaly type of the target anomaly region; performing anomaly marking processing on the user's target image data based on the anomaly information of the target anomaly region to obtain the user's image marking information; and generating the user's regional anomaly guidance report based on the user's regional anomaly status, the user's regional anomaly degree, and the user's image marking information, using a report guidance template.

[0094] In this embodiment, the terminal identifies the user's regional abnormality state and the degree of regional abnormality based on the abnormality type of the target abnormal region. Different abnormality types correspond to different regional abnormality states and different degrees of regional abnormality. The regional abnormality state is one of the specific states of the target abnormal region, including numerical abnormality, range abnormality, and change abnormality.

[0095] Then, based on the abnormal information of the target abnormal region, the terminal performs abnormal marking processing on the user's target image data to obtain the user's image marking information. Based on the user's regional abnormality status, the user's regional abnormality degree, and the user's image marking information, the terminal generates a regional abnormality guidance report for the user using a report guidance template. The user's image marking information includes the regional marker of the target abnormal region, the corresponding regional abnormality degree, the regional numerical distribution, and the regional abnormality status, thus facilitating medical staff's intuitive understanding of the target abnormal region and its abnormal information.

[0096] Based on the above scheme, by marking the abnormal regions of the target in terms of abnormal state, abnormal degree, and abnormality, the accuracy and comprehensiveness of anomaly identification in reconstructed images are improved. This further reduces the error rate of manual analysis and the problem of subjective observation bias, thereby improving the accuracy and comprehensiveness of anomaly identification.

[0097] The application also provides an example of image reconstruction based on deep learning, such as... Figure 2 As shown, the specific processing procedure includes the following steps:

[0098] Step S201: Obtain image data for different scanning methods of the user.

[0099] Step S202: Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method.

[0100] Step S203: Based on the regional structure data corresponding to each scanning mode, the image data of each scanning mode is processed by an image displacement registration procedure to obtain the sub-registered image corresponding to each scanning mode.

[0101] Step S204: Use the sub-registration images corresponding to all scanning methods as the user's registration image data.

[0102] Step S205: Perform grayscale truncation processing on the sub-registration image corresponding to the glucose metabolism scanning mode to obtain the grayscale truncation image corresponding to the glucose metabolism scanning mode.

[0103] Step S206: Identify the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, respectively. Based on the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, obtain the fluoroscopic scan image corresponding to the fluoroscopic scanning method and the metabolic scan image corresponding to the grayscale truncated image through a pixel normalization processing strategy.

[0104] Step S207: Extract the image data of the target layer range in the fluoroscopic scan image as the standardized fluoroscopic scan image corresponding to the fluoroscopic scan image, and extract the image data of the target layer range in the metabolic scan image as the standardized metabolic scan image corresponding to the metabolic scan image.

[0105] Step S208: The standardized fluoroscopic scan and the standardized metabolic scan are used as the user's standardized image data.

[0106] Step S209: In response to the information upload operation of the staff, obtain the random noise information of the standardized fluoroscopic scan image and the prompt word information of the standardized fluoroscopic scan image.

[0107] In step S210, the standardized perspective scan image, the random noise information of the standardized perspective scan image, the prompt word information of the standardized perspective scan image, and the standardized metabolic scan image are input into the image reconstruction model to generate the user's target image data.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this application also provides a deep learning-based image reconstruction apparatus for implementing the deep learning-based image reconstruction method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more deep learning-based image reconstruction apparatus embodiments provided below can be found in the limitations of the deep learning-based image reconstruction method described above, and will not be repeated here.

[0110] In one exemplary embodiment, such as Figure 3 As shown, a deep learning-based image reconstruction device is provided, comprising: an acquisition module 310, a processing module 320, and a construction module 330, wherein:

[0111] The acquisition module 310 is used to acquire image data for different scanning methods of the user;

[0112] The processing module 320 is used to perform image registration processing on the image data of each of the scanning methods to obtain the registered image data of the user, and to perform image optimization processing on the registered image data to obtain the standardized image data of the user.

[0113] The construction module 330 is used to construct the user's target image data based on the standardized image data and through an image reconstruction model.

[0114] Optionally, the processing module 320 is specifically used for:

[0115] Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method;

[0116] Based on the regional structure data corresponding to each of the scanning methods, the image data of each of the scanning methods are processed by an image displacement registration program to obtain the sub-registered images corresponding to each of the scanning methods.

[0117] The sub-registration images corresponding to all scanning methods are used as the user's registration image data.

[0118] Optionally, the processing module 320 is specifically used for:

[0119] The grayscale value of the sub-registration image corresponding to the glucose metabolism scanning mode is truncated to obtain the grayscale value truncated image corresponding to the glucose metabolism scanning mode.

[0120] The first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image are identified respectively. Based on the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, a pixel normalization processing strategy is used to obtain the fluoroscopic scan image corresponding to the fluoroscopic scanning method and the metabolic scan image corresponding to the grayscale truncated image.

[0121] Image data of the target layer range in the fluoroscopic scan is extracted and used as the standardized fluoroscopic scan corresponding to the fluoroscopic scan. Image data of the target layer range in the metabolic scan is also extracted and used as the standardized metabolic scan corresponding to the metabolic scan.

[0122] The standardized fluoroscopic scan and the standardized metabolic scan are used as the user's standardized image data.

[0123] Optionally, the building module 330 is specifically used for:

[0124] In response to the information upload operation of the staff, the system obtains the random noise information of the standardized fluoroscopic scan image and the prompt word information of the standardized fluoroscopic scan image.

[0125] The standardized perspective scan image, the random noise information of the standardized perspective scan image, the prompt word information of the standardized perspective scan image, and the standardized metabolic scan image are input into the image reconstruction model to generate the user's target image data.

[0126] Optionally, the device further includes:

[0127] The extraction module is used to extract the target abnormal region and the abnormal region feature data of the target abnormal region from the target image data;

[0128] The identification module is used to identify the anomaly type and anomaly information of the target anomaly region based on the anomaly region feature data of the target anomaly region through an image feature analysis model.

[0129] The generation module is used to generate a regional anomaly guidance report for the user based on the anomaly type of the target anomaly region and the anomaly information of the target anomaly region.

[0130] Optionally, the generation module is specifically used for:

[0131] Based on the anomaly type of the target anomaly region, identify the user's regional anomaly state and the degree of regional anomaly.

[0132] Based on the anomaly information of the target anomaly region, anomaly marking processing is performed on the user's target image data to obtain the user's image marking information. Based on the user's regional anomaly status, the user's regional anomaly degree, and the user's image marking information, a regional anomaly guidance report for the user is generated through a report guidance template.

[0133] The modules in the aforementioned deep learning-based image reconstruction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0134] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a deep learning-based image reconstruction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0135] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a deep learning-based image reconstruction method.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a deep learning-based image reconstruction method.

[0138] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of a deep learning-based image reconstruction method.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0140] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A deep learning-based image reconstruction method, characterized in that, The method includes: Acquire image data for users using different scanning methods; Image registration processing is performed on the image data of each of the aforementioned scanning methods to obtain the registered image data of the user. Then, image optimization processing is performed on the registered image data to obtain the standardized image data of the user. The standardized image data includes standardized fluoroscopic scan images and standardized metabolic scan images. In response to the information upload operation of the staff, the random noise information of the standardized perspective scan image and the prompt word information of the standardized perspective scan image are obtained; the random noise information is the image data corresponding to the noise pixels in the standardized perspective scan image, and the prompt word information includes the image data of the key point pixels in the standardized perspective scan image, or the edge image data in the standardized perspective scan image. Input the standardized fluoroscopic scan image, the random noise information of the standardized fluoroscopic scan image, the prompt word information of the standardized fluoroscopic scan image, and the standardized metabolic scan image into the Control-Net branch; After the Control-Net branch performs processing, condition information is generated and then transmitted to the Stable Diffusion master control model. The Stable Diffusion master control model is controlled to process the prompt information and the condition information of the Control-Net branch to generate the user's target image data; the target image data is a PET image that is consistent with the functional information and anatomical structure of the CT image.

2. The method according to claim 1, characterized in that, The step of performing image registration processing on the image data of each of the aforementioned scanning methods to obtain the registered image data of the user includes: Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method; Based on the regional structure data corresponding to each of the scanning methods, the image data of each of the scanning methods are processed by an image displacement registration program to obtain the sub-registered images corresponding to each of the scanning methods. The sub-registration images corresponding to all scanning methods are used as the user's registration image data.

3. The method according to claim 2, characterized in that, The scanning methods include fluoroscopic scanning and glucose metabolism scanning. The process of optimizing the registered image data to obtain the user's standardized image data includes: The grayscale value of the sub-registration image corresponding to the glucose metabolism scanning mode is truncated to obtain the grayscale value truncated image corresponding to the glucose metabolism scanning mode. The first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image are identified respectively. Based on the first image pixel data corresponding to the sub-registered image of the fluoroscopic scanning method and the second image pixel data corresponding to the grayscale truncated image, a pixel normalization processing strategy is used to obtain the fluoroscopic scan image corresponding to the fluoroscopic scanning method and the metabolic scan image corresponding to the grayscale truncated image. Image data of the target layer range in the fluoroscopic scan is extracted and used as the standardized fluoroscopic scan corresponding to the fluoroscopic scan. Image data of the target layer range in the metabolic scan is also extracted and used as the standardized metabolic scan corresponding to the metabolic scan. The standardized fluoroscopic scan and the standardized metabolic scan are used as the user's standardized image data.

4. The method according to claim 1, characterized in that, After constructing the user's target image data based on the standardized image data using an image reconstruction model, the process further includes: Extract the target anomaly region from the target image data, and extract the anomaly region feature data of the target anomaly region; Based on the abnormal region feature data of the target abnormal region, the abnormal type and abnormal information of the target abnormal region are identified through an image feature analysis model. Based on the anomaly type and anomaly information of the target anomaly region, a regional anomaly guidance report is generated for the user.

5. The method according to claim 4, characterized in that, The process of generating a regional anomaly guidance report for the user based on the anomaly type and anomaly information of the target anomaly region includes: Based on the anomaly type of the target anomaly region, identify the user's regional anomaly state and the degree of regional anomaly of the user; Based on the anomaly information of the target anomaly region, anomaly marking processing is performed on the user's target image data to obtain the user's image marking information. Based on the user's regional anomaly status, the user's regional anomaly degree, and the user's image marking information, a regional anomaly guidance report for the user is generated through a report guidance template.

6. An image reconstruction device based on deep learning, characterized in that, The device includes: The acquisition module is used to acquire image data for different scanning methods used by the user; The processing module is used to perform image registration processing on the image data of each of the scanning methods to obtain the registered image data of the user, and to perform image optimization processing on the registered image data to obtain the standardized image data of the user; the standardized image data includes standardized fluoroscopic scan images and standardized metabolic scan images. A construction module is used to respond to the information upload operation of the staff, and to obtain the random noise information and prompt word information of the standardized fluoroscopic scan image. The random noise information is the image data corresponding to the noise pixels in the standardized fluoroscopic scan image, and the prompt word information includes the image data of key point pixels in the standardized fluoroscopic scan image, or the edge image data in the standardized fluoroscopic scan image. The standardized fluoroscopic scan image, the random noise information of the standardized fluoroscopic scan image, the prompt word information of the standardized fluoroscopic scan image, and the standardized metabolic scan image are input into the Control-Net branch. After processing by the Control-Net branch, condition information is generated and passed to the Stable Diffusion master control model. The Stable Diffusion master control model is controlled to process the prompt word information and the condition information of the Control-Net branch to generate the user's target image data. The target image data is a PET image consistent with the functional information and anatomical structure of the CT image.

7. The apparatus according to claim 6, characterized in that, The processing module is specifically used for: Based on the image data of each scanning method, identify the region structure data corresponding to each scanning method; Based on the regional structure data corresponding to each of the scanning methods, the image data of each of the scanning methods are processed by an image displacement registration program to obtain the sub-registered images corresponding to each of the scanning methods. The sub-registration images corresponding to all scanning methods are used as the user's registration image data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images