Image reconstruction method and device based on deep learning

Abstract

The invention relates to an image reconstruction method and device based on deep learning. The method comprises the following steps: acquiring image data of different scanning modes for a user; performing image registration processing on the image data of each scanning mode to obtain registration image data of the user, and performing image and image optimization processing on the registration image data to obtain standardized image data of the user; and based on the standardized image data, constructing target image data of the user through an image reconstruction model. By adopting the method, the imaging precision of the Parkinson's early diagnosis image can be improved.

Description

Technical Field

[0001] The present application relates to the fields of image reconstruction and medical image technology, and particularly to an image reconstruction method and device based on deep learning. Background Art

[0002] Medical images are important tools for disease diagnosis. CT (Computed Tomography) and PET (Positron Emission Tomography) are two commonly used imaging techniques in the diagnosis of Parkinson's disease. CT generates clear images of anatomical structures through X-ray scanning and is widely used in the diagnosis of bone and soft tissue lesions, but it cannot reflect the metabolic activities of tissues. PET generates images reflecting metabolic and functional information through tracer injection. In particular, 18F-FDG PET can quantitatively detect cerebral glucose metabolism, and its abnormal changes often precede the abnormalities seen in anatomical structure imaging. Therefore, how to improve the accuracy of image acquisition and the comprehensiveness of detailed structure feedback is the current research focus of medical image technology.

[0003] Traditional technical solutions use deep learning methods for image reconstruction and fusion. Traditional multi-modal image fusion relies highly on data registration and lacks precise control over the consistency between functional information and anatomical structures. This makes it difficult to accurately detect early lesions of Parkinson's disease, especially minor structural changes, and also provides limited support for the quantitative analysis of PET images, resulting in poor imaging accuracy for early diagnosis images of Parkinson's disease. Summary of the Invention

[0004] Based on this, it is necessary to provide an image reconstruction method, device, computer device, computer-readable storage medium, and computer program product based on deep learning for the above technical problems.

[0005] In a first aspect, the present application provides an image reconstruction method based on deep learning, including:

[0006] Obtaining image data of different scanning methods of a user;

[0007] Performing image registration processing on the image data of each scanning method to obtain the registered image data of the user, and performing image optimization processing on the registered image data to obtain the standardized image data of the user;

[0008] Based on the standardized image data, constructing the target image data of the user through an image reconstruction model.

[0009] Optionally, the performing image registration processing on the image data of each scanning method to obtain the registered image data of the user includes:

[0010] Identify the regional structure data corresponding to each of the scanning methods based on the image data of each of the scanning methods;

[0011] Based on the regional structure data corresponding to each of the scanning methods, perform image registration processing on the image data of each of the scanning methods through an image displacement registration program to obtain sub-registered images corresponding to each of the scanning methods;

[0012] Use the sub-registered images corresponding to all the scanning methods as the registered image data of the user.

[0013] Optionally, the scanning methods include a fluoroscopy scanning method and a glucose metabolism scanning method. The image optimization process for the registered image data to obtain the standardized image data of the user includes:

[0014] Perform grayscale value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a grayscale value truncation map corresponding to the glucose metabolism scanning method;

[0015] Respectively identify the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the grayscale value truncation map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the grayscale value truncation map, obtain a fluoroscopy scan map corresponding to the fluoroscopy scanning method and a metabolic scan map corresponding to the grayscale value truncation map through a pixel normalization processing strategy;

[0016] Extract the image data of the target layer range in the fluoroscopy scan map as the standardized fluoroscopy scan map corresponding to the fluoroscopy scan map, and extract the image data of the target layer range in the metabolic scan map as the standardized metabolic scan map corresponding to the metabolic scan map;

[0017] Use the standardized fluoroscopy scan map and the standardized metabolic scan map as the standardized image data of the user.

[0018] Optionally, the construction of the target image data of the user through an image reconstruction model based on the standardized image data includes:

[0019] In response to the information upload operation of the staff, obtain the random noise information of the standardized fluoroscopy scan map and the prompt word information of the standardized fluoroscopy scan map;

[0020] Input the standardized fluoroscopy scan map, the random noise information of the standardized fluoroscopy scan map, the prompt word information of the standardized fluoroscopy scan map, and the standardized metabolic scan map into the image reconstruction model to generate the target image data of the user.

[0021] Optionally, after constructing the target image data of the user based on the standardized image data through an image reconstruction model, the method further includes:

[0022] Extracting the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region;

[0023] Based on the abnormal region feature data of the target abnormal region, identifying the abnormal type of the target abnormal region and the abnormal information of the target abnormal region through an image feature analysis model;

[0024] Generating a regional abnormal guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region.

[0025] Optionally, the generating a regional abnormal guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region includes:

[0026] Based on the abnormal type of the target abnormal region, identifying the regional abnormal status of the user and the regional abnormal degree of the user;

[0027] Based on the abnormal information of the target abnormal region, performing abnormal marking processing on the target image data of the user to obtain the image marking information of the user, and generating a regional abnormal guidance report for the user through a report guidance template based on the regional abnormal status of the user, the regional abnormal degree of the user, and the image marking information of the user.

[0028] In a second aspect, the present application further provides an image reconstruction device based on deep learning, including:

[0029] An acquisition module, configured to acquire image data of different scanning methods of a user;

[0030] A processing module, configured to perform image registration processing on the image data of each scanning method to obtain the registered image data of the user, and perform image optimization processing on the registered image data to obtain the standardized image data of the user;

[0031] A construction module, configured to construct the target image data of the user based on the standardized image data through an image reconstruction model.

[0032] Optionally, the processing module is specifically configured to:

[0033] Based on the image data of each scanning method, identify the regional structure data corresponding to each scanning method;

[0034] Based on the regional structure data corresponding to each of the scanning methods, through an image displacement registration program, perform image registration processing on the image data of each of the scanning methods to obtain sub-registered images corresponding to each of the scanning methods;

[0035] Use the sub-registered images corresponding to all scanning methods as the registered image data of the user.

[0036] Optionally, the processing module is specifically configured to:

[0037] Perform gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a gray value truncated map corresponding to the glucose metabolism scanning method;

[0038] Respectively identify the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncated map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncated map, through a pixel normalization processing strategy, obtain a fluoroscopy scan map corresponding to the fluoroscopy scanning method and a metabolic scan map corresponding to the gray value truncated map;

[0039] Extract the image data of the target layer range in the fluoroscopy scan map as the standardized fluoroscopy scan map corresponding to the fluoroscopy scan map, and extract the image data of the target layer range in the metabolic scan map as the standardized metabolic scan map corresponding to the metabolic scan map;

[0040] Use the standardized fluoroscopy scan map and the standardized metabolic scan map as the standardized image data of the user.

[0041] Optionally, the construction module is specifically configured to:

[0042] In response to the information uploading operation of the staff, obtain the random noise information of the standardized fluoroscopy scan map and the prompt word information of the standardized fluoroscopy scan map;

[0043] Input the standardized fluoroscopy scan map, the random noise information of the standardized fluoroscopy scan map, the prompt word information of the standardized fluoroscopy scan map, and the standardized metabolic scan map into an image reconstruction model to generate the target image data of the user.

[0044] Optionally, the device further includes:

[0045] An extraction module, configured to extract the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region;

[0046] An identification module, configured to identify the abnormal type of the target abnormal area and the abnormal information of the target abnormal area through an image feature analysis model based on the abnormal area feature data of the target abnormal area;

[0047] A generation module, configured to generate a regional abnormal guidance report for the user based on the abnormal type of the target abnormal area and the abnormal information of the target abnormal area.

[0048] Optionally, the generation module is specifically configured to:

[0049] Identify the regional abnormal status of the user and the degree of regional abnormality of the user based on the abnormal type of the target abnormal area;

[0050] Perform abnormal marking processing on the target image data of the user based on the abnormal information of the target abnormal area to obtain the image marking information of the user, and generate the regional abnormal guidance report for the user through a report guidance template based on the regional abnormal status of the user, the degree of regional abnormality of the user, and the image marking information of the user.

[0051] In a third aspect, the present application provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the first aspects are implemented.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium. A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0053] In a fifth aspect, the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0054] The above-mentioned image reconstruction method and device based on deep learning obtain image data of different scanning methods for a user; perform image registration processing on the image data of each scanning method to obtain the registered image data of the user, and perform image optimization processing on the registered image data to obtain the standardized image data of the user; based on the standardized image data, construct the target image data of the user through an image reconstruction model. This solution combines CT images and PET images, performs registration and image optimization processing on the two images, so as to ensure both the balance of calculation efficiency and the retention of image details, and can accurately obtain the image data corresponding to the effective regions in different scanning methods, and then perform image reconstruction processing, so that the generated target image data contains both accurate structural information and image data reflecting the structural image, as well as the standardized image data of the images obtained by different scanning methods, so that the generated target image can comprehensively and comprehensively reflect the scanning information of each scan of the user. Among them, when the target image is used in the generation process of images for early Parkinson's diagnosis, it can comprehensively and accurately reflect the microstructural changes and lesion conditions of Parkinson's disease, thus effectively improving the imaging accuracy of images for early Parkinson's diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of an image reconstruction method based on deep learning in an embodiment;

[0057] Figure 2 It is a flowchart of an image reconstruction example based on deep learning in an embodiment;

[0058] Figure 3 It is a structural block diagram of an image reconstruction device based on deep learning in an embodiment;

[0059] Figure 4 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0061] The image reconstruction method based on deep learning provided by the embodiments of the present application can be applied to an intelligent control system for image reconstruction based on deep learning. This system can be applied to a terminal, which can be, but is not limited to, various personal computers, laptop computers, mid-range computers, etc. Among them, the terminal combines CT images and PET images, registers the two images, and performs image optimization processing, so as to ensure both the balance of computing efficiency and the retention of image details, and can accurately obtain the image data corresponding to the effective regions in different scanning methods, and then performs image reconstruction processing, so that the generated target image data contains both accurate structural information and image data reflecting the structural image, and also contains the standardized image data of the images obtained by different scanning methods, so that the generated target image can comprehensively and comprehensively reflect all the scanning information of the user's scan. Among them, when the target image is used in the generation process of images for early Parkinson's disease diagnosis, it can comprehensively and accurately reflect the microscopic structural changes and lesion conditions of Parkinson's disease, thus effectively improving the imaging accuracy of early Parkinson's disease diagnosis images.

[0062] In an exemplary embodiment, as Figure 1 shown, a method for image reconstruction based on deep learning is provided. Taking the application of this method to a terminal as an example, it includes the following steps S101 to S103. Among them:

[0063] Step S101, obtain the image data of different scanning methods for the user.

[0064] In this embodiment, the terminal performs regional scanning on the target region where the user needs to generate disease images through different scanning methods, and obtains the image data of different scanning methods for the user. Among them, the target region where the disease images need to be generated includes, but is not limited to, the brain region for detecting neurodegenerative diseases such as Parkinson's disease and Alzheimer's disease, the heart region for detecting cardiovascular diseases, and other tumor generation regions for detecting diseases such as tumors. And the scanning methods include, but are not limited to, CT scanning methods and PET scanning methods.

[0065] Step S102, perform image registration processing on the image data of each scanning method to obtain the registered image data of the user, and perform image optimization processing on the registered image data to obtain the standardized image data of the user.

[0066] In this embodiment, the terminal performs image registration processing on the image data of each scanning method to obtain the registered image data of the user, and performs image optimization processing on the registered image data to obtain the standardized image data of the user. Among them, the image registration processing is a method of performing displacement registration on the scanned images corresponding to different scanning methods. And the image optimization method includes image optimization methods such as gray value truncation, pixel normalization, and slice cropping. The specific registration process and image optimization process will be described in detail later.

[0067] Step S103, based on the standardized image data, construct the target image data of the user through the image reconstruction model.

[0068] In this embodiment, the terminal constructs the target image data of the user based on the standardized image data through the image reconstruction model. Among them, the image reconstruction model is the Stable Diffusion model of Control-Net designed in this solution. This model is based on the Stable Diffusion model and adds a Control-Net branch to achieve precise control. The specific construction process will be described in detail later.

[0069] Based on the above solution, by combining CT images and PET images, and performing registration and image optimization processing on the two images, it is possible to ensure both balanced calculation efficiency and retention of image details, and accurately obtain the image data corresponding to the effective regions in different scanning methods. Then, image reconstruction processing is performed, so that the generated target image data contains both accurate structural information and image data reflecting the structural image, as well as the standardized image data of the images obtained by different scanning methods. Therefore, the generated target image can comprehensively and comprehensively reflect all the scanning information of the user's scan. Among them, when the target image is used in the generation process of Parkinson's early diagnosis images, it can comprehensively and accurately reflect the minute structural changes and lesion conditions of Parkinson's disease, thereby effectively improving the imaging accuracy of Parkinson's early diagnosis images.

[0070] Optionally, performing image registration processing on the image data of each scanning method to obtain the registered image data of the user includes: based on the image data of each scanning method, identifying the regional structure data corresponding to each scanning method; based on the regional structure data corresponding to each scanning method, through the image displacement registration program, performing image registration processing on the image data of each scanning method to obtain the sub-registered images corresponding to each scanning method; using the sub-registered images corresponding to all scanning methods as the registered image data of the user.

[0071] In this embodiment, the terminal identifies the regional structure data corresponding to each scanning method based on the image data of each scanning method. The regional structure data is the image structure data of the target region corresponding to the image of each scanning method, such as brain edge structure data, heart edge contour structure data, etc.

[0072] Then, based on the regional structure data corresponding to each scanning method, the terminal performs image registration processing on the image data of each scanning method through an image displacement registration program to obtain sub-registered images corresponding to each scanning method. The image displacement registration program can be the registration program corresponding to the Elastix software.

[0073] Finally, the terminal uses the sub-registered images corresponding to all scanning methods as the registered image data of the user.

[0074] Based on the above solution, by performing structural displacement registration on the image data of different scanning methods, the displacement deviation caused by the scanning error of the imaging device is adjusted. The registration ensures the precise alignment of the anatomical structures in the two modalities, improving the accuracy of image registration and fusion.

[0075] Optionally, the scanning methods include a fluoroscopy scanning method and a glucose metabolism scanning method. The registered image data is subjected to image optimization processing to obtain the standardized image data of the user, including: performing gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a gray value truncation map corresponding to the glucose metabolism scanning method; respectively identifying the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, through a pixel normalization processing strategy, obtaining a fluoroscopy scan map corresponding to the fluoroscopy scanning method and a metabolism scan map corresponding to the gray value truncation map; extracting the image data within the target layer range in the fluoroscopy scan map as the standardized fluoroscopy scan map corresponding to the fluoroscopy scan map, and extracting the image data within the target layer range in the metabolism scan map as the standardized metabolism scan map corresponding to the metabolism scan map; using the standardized fluoroscopy scan map and the standardized metabolism scan map as the standardized image data of the user.

[0076] In this embodiment, the terminal performs gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a gray value truncation map corresponding to the glucose metabolism scanning method. Specifically, the terminal performs gray value truncation on the PET image, limits the intensity value within a reasonable range (for example, within 12000), and performs normalization processing to make the data distribution adapt to the model requirements. The result is output as a processed gray value histogram for quality verification.

[0077] Then, the terminal respectively identifies the first image pixel data corresponding to the sub-registration image in the perspective scanning mode and the second image pixel data corresponding to the gray value truncation map. Among them, the identification method of the image pixel data corresponding to each image is to respectively perform pixel value identification on each image through a pixel data identification program preset in the terminal, obtain the pixel value distribution information of each image, and use the pixel value distribution information of each image as the image pixel data corresponding to each image. This pixel data identification program is a program designed for this solution to identify image pixels. Specifically, the terminal adjusts the voxel sizes of the CT and PET images to unify the spatial resolution, and at the same time compresses the image size to 256×256 pixels to balance the calculation efficiency and the retention of image details.

[0078] Then, based on the first image pixel data corresponding to the sub-registration image in the perspective scanning mode and the second image pixel data corresponding to the gray value truncation map, the terminal obtains the perspective scan map corresponding to the perspective scanning mode and the metabolic scan map corresponding to the gray value truncation map through a pixel normalization processing strategy.

[0079] The terminal extracts the image data within the target layer range from the perspective scan map as the standardized perspective scan map corresponding to the perspective scan map, and extracts the image data within the target layer range from the metabolic scan map as the standardized metabolic scan map corresponding to the metabolic scan map. Among them, the extracted target layer is the image data corresponding to the image layer that removes the irrelevant first 10 layers and the last 10 layers of slices and only retains the middle effective area.

[0080] Finally, the terminal uses the standardized perspective scan map and the standardized metabolic scan map as the user's standardized image data.

[0081] Based on the above solution, by optimizing the image, not only the representation degree of the image feature data is improved, but also the image difference between the pictures corresponding to each scanning mode can be effectively reduced, thereby improving the image fusion efficiency and fusion accuracy.

[0082] Optionally, based on the standardized image data, through an image reconstruction model, the user's target image data is constructed, including: in response to the information upload operation of the staff, obtaining the random noise information of the standardized perspective scan map and the prompt word information of the standardized perspective scan map; inputting the standardized perspective scan map, the random noise information of the standardized perspective scan map, the prompt word information of the standardized perspective scan map, and the standardized metabolic scan map into the image reconstruction model to generate the user's target image data.

[0083] In this embodiment, the terminal responds to the information uploading operation of the staff member to obtain the random noise information of the standardized fluoroscopic scan image and the prompt word information of the standardized fluoroscopic scan image. Specifically, the obtained random noise information is the image data corresponding to the noise pixels in the standardized fluoroscopic scan image, and the prompt word information includes the image data of the key point pixels in the standardized fluoroscopic scan image, or the edge image data in the standardized fluoroscopic scan image, etc.

[0084] Finally, the terminal inputs the standardized fluoroscopic scan image, the random noise information of the standardized fluoroscopic scan image, the prompt word information of the standardized fluoroscopic scan image, and the standardized metabolic scan image into the image reconstruction model to generate the target image data of the user. Specifically, the processed CT image, random noise, and prompt words (such as key points or edge maps) are used as inputs and enter the Control-Net branch.

[0085] Then, the terminal controls Control-Net to process the input data, generate conditional information, and transfer it to the StableDiffusion main control model. Specifically, the trainable part: learns the deep representation of the control variables to optimize the generation effect. The locked part: retains the original generation ability of Stable Diffusion to ensure the stability of the model.

[0086] After that, the terminal controls Stable Diffusion to combine the prompt words and the conditional output of Control-Net to generate a PET image that is consistent with the functional information and anatomical structure of the CT image.

[0087] Based on the above solution, the Stable Diffusion image reconstruction model based on Control-Net constructed by this solution generates a PET image that is consistent with the functional information and anatomical structure of the CT image, thereby ensuring that the fused image contains both the structural data consistent with the functional information and anatomical structure of the CT image and the characteristics of the PET image that reflect metabolic and functional information, improving the comprehensiveness of the feature representation of the fused image and the accuracy of the display of image details.

[0088] Optionally, after constructing the target image data of the user through the image reconstruction model based on the standardized image data, it further includes: extracting the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region; based on the abnormal region feature data of the target abnormal region, identifying the abnormal type of the target abnormal region and the abnormal information of the target abnormal region through the image feature analysis model; generating a regional abnormal guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region.

[0089] In this embodiment, the terminal extracts the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region. Among them, the abnormal region feature data is the target image data of the target region, and the image feature data extracted by the feature extraction network based on the deep learning algorithm. This image feature data is, for example, the change information of the metabolic activity in the brain region.

[0090] Then, based on the abnormal region feature data of the target abnormal region, the terminal uses an image feature analysis model to identify the abnormal type of the target abnormal region and the abnormal information of the target abnormal region. Among them, the image feature analysis model is a convolutional neural network based on the deep learning algorithm. The abnormal types of the target abnormal region extracted are, for example, metabolic abnormal types, brain region numerical abnormal types, brain region structure abnormalities, and other abnormal types. And the abnormal information of the target abnormal region includes the metabolic numerical range, brain region numerical range, brain region structure data range, etc.

[0091] Finally, based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region, the terminal generates a regional abnormal guidance report for the user. Among them, this regional abnormal guidance report is used to guide the staff to the abnormal range, abnormal status, and abnormal degree in the target region of the user. The specific generation process will be described in detail later.

[0092] Based on the above solution, by combining the reconstructed image and performing quantitative analysis using deep learning technology, it is possible to guide medical staff to identify the lesion area earlier and more accurately, especially in the early stage of Parkinson's disease, thus improving the guidance accuracy and efficiency for medical staff.

[0093] Optionally, generating a regional abnormal guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region includes: identifying the regional abnormal status of the user and the regional abnormal degree of the user based on the abnormal type of the target abnormal region; performing abnormal marking processing on the target image data of the user based on the abnormal information of the target abnormal region to obtain the image marking information of the user, and generating a regional abnormal guidance report for the user through a report guidance template based on the regional abnormal status of the user, the regional abnormal degree of the user, and the image marking information of the user.

[0094] In this embodiment, the terminal identifies the regional abnormal status of the user and the regional abnormal degree of the user based on the abnormal type of the target abnormal region. Among them, different abnormal types correspond to different regional abnormal statuses, and different abnormal types also correspond to different regional abnormal degrees. Among them, the regional abnormal status is one specific status among the numerical abnormal status, range abnormal status, change abnormal status, etc. of the target abnormal region.

[0095] Then, based on the abnormal information of the target abnormal area, the terminal performs abnormal marking processing on the target image data of the user to obtain the image marking information of the user, and generates the regional abnormal guidance report of the user through the report guidance template based on the regional abnormal state of the user, the regional abnormal degree of the user, and the image marking information of the user. Among the image marking information of the user, it includes the regional marking of the target abnormal area, the regional abnormal degree corresponding to the abnormal area, the regional numerical distribution, and the marking information such as the regional abnormal state, so as to facilitate medical staff to intuitively understand the target abnormal area and the abnormal information of the target abnormal area.

[0096] Based on the above solution, by marking the target abnormal area from aspects such as abnormal state, abnormal degree, and abnormality, the accuracy and comprehensiveness of abnormal recognition of the reconstructed image are improved, thereby further reducing the error rate of manual analysis and the problem of subjective observation deviation, and improving the accuracy and comprehensiveness of abnormal recognition.

[0097] The application also provides an example of image reconstruction based on deep learning, as Figure 2 shown. The specific processing process includes the following steps:

[0098] Step S201, obtain the image data of different scanning methods of the user.

[0099] Step S202, based on the image data of each scanning method, identify the regional structure data corresponding to each scanning method.

[0100] Step S203, based on the regional structure data corresponding to each scanning method, through the image displacement registration program, perform image registration processing on the image data of each scanning method to obtain the sub-registered images corresponding to each scanning method.

[0101] Step S204, use the sub-registered images corresponding to all scanning methods as the registered image data of the user.

[0102] Step S205, perform gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain the gray value truncation map corresponding to the glucose metabolism scanning method.

[0103] Step S206, respectively identify the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, through the pixel normalization processing strategy, obtain the fluoroscopy scan map corresponding to the fluoroscopy scanning method and the metabolic scan map corresponding to the gray value truncation map.

[0104] Step S207, extract the image data of the target layer range in the perspective scan image as the standardized perspective scan image corresponding to the perspective scan image, and extract the image data of the target layer range in the metabolic scan image as the standardized metabolic scan image corresponding to the metabolic scan image.

[0105] Step S208, use the standardized perspective scan image and the standardized metabolic scan image as the user's standardized image data.

[0106] Step S209, in response to the information uploading operation of the staff, obtain the random noise information of the standardized perspective scan image and the prompt word information of the standardized perspective scan image.

[0107] Step S210, input the standardized perspective scan image, the random noise information of the standardized perspective scan image, the prompt word information of the standardized perspective scan image, and the standardized metabolic scan image into the image reconstruction model to generate the user's target image data.

[0108] It should be understood that although each step in the flowcharts involved in the above-described embodiments is shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiments of the present application also provide a deep learning-based image reconstruction device for implementing the above-mentioned deep learning-based image reconstruction method. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more of the following embodiments of the deep learning-based image reconstruction device can refer to the limitations on the deep learning-based image reconstruction method in the above text, and will not be repeated here.

[0110] In an exemplary embodiment, as Figure 3 shown, a deep learning-based image reconstruction device is provided, including: an acquisition module 310, a processing module 320, and a construction module 330, where:

[0111] The acquisition module 310 is configured to acquire the image data of different scanning methods for the user;

[0112] A processing module 320 is configured to perform image registration processing on the image data of each of the scanning methods to obtain the registered image data of the user, and perform image optimization processing on the registered image data to obtain the standardized image data of the user;

[0113] A construction module 330 is configured to construct the target image data of the user based on the standardized image data through an image reconstruction model.

[0114] Optionally, the processing module 320 is specifically configured to:

[0115] Based on the image data of each of the scanning methods, identify the regional structure data corresponding to each of the scanning methods;

[0116] Based on the regional structure data corresponding to each of the scanning methods, perform image registration processing on the image data of each of the scanning methods through an image displacement registration program to obtain sub-registered images corresponding to each of the scanning methods;

[0117] Use the sub-registered images corresponding to all the scanning methods as the registered image data of the user.

[0118] Optionally, the processing module 320 is specifically configured to:

[0119] Perform gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a gray value truncation map corresponding to the glucose metabolism scanning method;

[0120] Respectively identify the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, obtain a fluoroscopy scan map corresponding to the fluoroscopy scanning method and a metabolic scan map corresponding to the gray value truncation map through a pixel normalization processing strategy;

[0121] Extract the image data of the target layer range in the fluoroscopy scan map as the standardized fluoroscopy scan map corresponding to the fluoroscopy scan map, and extract the image data of the target layer range in the metabolic scan map as the standardized metabolic scan map corresponding to the metabolic scan map;

[0122] Use the standardized fluoroscopy scan map and the standardized metabolic scan map as the standardized image data of the user.

[0123] Optionally, the construction module 330 is specifically configured to:

[0124] In response to the information upload operation of the staff, obtain the random noise information of the standardized fluoroscopy scan image and the prompt word information of the standardized fluoroscopy scan image;

[0125] Input the standardized fluoroscopy scan image, the random noise information of the standardized fluoroscopy scan image, the prompt word information of the standardized fluoroscopy scan image, and the standardized metabolic scan image into an image reconstruction model to generate the target image data of the user.

[0126] Optionally, the device further includes:

[0127] An extraction module, configured to extract the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region;

[0128] An identification module, configured to identify the abnormal type of the target abnormal region and the abnormal information of the target abnormal region through an image feature analysis model based on the abnormal region feature data of the target abnormal region;

[0129] A generation module, configured to generate a regional abnormality guidance report for the user based on the abnormal type of the target abnormal region and the abnormal information of the target abnormal region.

[0130] Optionally, the generation module is specifically configured to:

[0131] Based on the abnormal type of the target abnormal region, identify the regional abnormal state of the user and the degree of regional abnormality of the user;

[0132] Based on the abnormal information of the target abnormal region, perform abnormal marking processing on the target image data of the user to obtain the image marking information of the user, and based on the regional abnormal state of the user, the degree of regional abnormality of the user, and the image marking information of the user, generate a regional abnormality guidance report for the user through a report guidance template.

[0133] Each module in the above image reconstruction device based on deep learning can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above modules.

[0134] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 4As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for image reconstruction based on deep learning. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0135] Those skilled in the art can understand that Figure 4 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0136] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps of a method for image reconstruction based on deep learning are implemented.

[0137] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps of a method for image reconstruction based on deep learning are implemented.

[0138] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps of a method for image reconstruction based on deep learning are implemented.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0140] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0141] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0142] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image reconstruction method based on deep learning, characterized in that, The method includes: Obtaining image data of different scanning methods of the user; Performing image registration processing on the image data of each scanning method to obtain the registered image data of the user, and performing image optimization processing on the registered image data to obtain the standardized image data of the user; Based on the standardized image data, constructing the target image data of the user through an image reconstruction model.

2. The method according to claim 1, wherein The performing image registration processing on the image data of each scanning method to obtain the registered image data of the user includes: Based on the image data of each scanning method, identifying the regional structure data corresponding to each scanning method; Based on the regional structure data corresponding to each scanning method, through an image displacement registration program, performing image registration processing on the image data of each scanning method to obtain sub-registered images corresponding to each scanning method; Taking the sub-registered images corresponding to all scanning methods as the registered image data of the user.

3. The method according to claim 2, wherein The scanning methods include a fluoroscopy scanning method and a glucose metabolism scanning method. The performing image optimization processing on the registered image data to obtain the standardized image data of the user includes: Performing gray value truncation processing on the sub-registered image corresponding to the glucose metabolism scanning method to obtain a gray value truncation map corresponding to the glucose metabolism scanning method; Respectively identifying the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, and based on the first image pixel data corresponding to the sub-registered image of the fluoroscopy scanning method and the second image pixel data corresponding to the gray value truncation map, through a pixel normalization processing strategy, obtaining a fluoroscopy scan map corresponding to the fluoroscopy scanning method and a metabolism scan map corresponding to the gray value truncation map; Extracting the image data of the target layer range in the fluoroscopy scan map as the standardized fluoroscopy scan map corresponding to the fluoroscopy scan map, and extracting the image data of the target layer range in the metabolism scan map as the standardized metabolism scan map corresponding to the metabolism scan map; Taking the standardized fluoroscopy scan map and the standardized metabolism scan map as the standardized image data of the user.

4. The method according to claim 3, wherein The constructing the target image data of the user through an image reconstruction model based on the standardized image data includes: In response to the information uploading operation of the staff, obtaining the random noise information of the standardized fluoroscopy scan map and the prompt word information of the standardized fluoroscopy scan map; Inputting the standardized fluoroscopy scan map, the random noise information of the standardized fluoroscopy scan map, the prompt word information of the standardized fluoroscopy scan map, and the standardized metabolism scan map into the image reconstruction model to generate the target image data of the user.

5. The method according to claim 1, wherein After constructing the target image data of the user through an image reconstruction model based on the standardized image data, it further includes: Extracting the target abnormal region of the target image data and the abnormal region feature data of the target abnormal region; Based on the abnormal area feature data of the target abnormal area, through an image feature analysis model, identify the abnormal type of the target abnormal area and the abnormal information of the target abnormal area; Based on the abnormal type of the target abnormal area and the abnormal information of the target abnormal area, generate a regional abnormal guidance report for the user.

6. The method according to claim 5, characterized in that, The generating the regional abnormal guidance report for the user based on the abnormal type of the target abnormal area and the abnormal information of the target abnormal area includes: Based on the abnormal type of the target abnormal area, identify the regional abnormal status of the user and the regional abnormal degree of the user; Based on the abnormal information of the target abnormal area, perform abnormal marking processing on the target image data of the user to obtain the image marking information of the user, and based on the regional abnormal status of the user, the regional abnormal degree of the user, and the image marking information of the user, generate the regional abnormal guidance report for the user through a report guidance template.

7. An image reconstruction device based on deep learning, characterized in that, The device includes: An acquisition module, configured to acquire image data of different scanning methods for the user; A processing module, configured to perform image registration processing on the image data of each scanning method to obtain the registered image data of the user, and perform image optimization processing on the registered image data to obtain the standardized image data of the user; A construction module, configured to construct the target image data of the user through an image reconstruction model based on the standardized image data.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images