Super-resolution reconstruction network model training method and scanned image processing method

Abstract

The present invention discloses a training method for a super-resolution reconstruction network model and a scanned image processing method. The training method includes: obtaining a low-resolution training image and a corresponding high-resolution training image; inputting the low-resolution training image into the super-resolution reconstruction network model to be trained; performing image reconstruction processing on the low-resolution training image using the super-resolution reconstruction network model to obtain a super-resolution training image corresponding to the low-resolution training image; calculating a first loss between the super-resolution and high-resolution training images; identifying a focus region from the super-resolution training image and the same focus region from the high-resolution training image, calculating a second loss between the super-resolution and high-resolution training images of the focus region; and training the super-resolution reconstruction network model based on the first and second losses. The present invention can flexibly select image detail restoration, significantly improving the restoration effect of the image focus region.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a training method for a super-resolution reconstruction network model and a scanned image processing method. Background Art

[0002] Morphological magnetic resonance (MR) analysis is an important tool in the field of neuroimaging for diagnosing brain diseases and studying brain development. Low-resolution images restrict the field of view and provide limited pathological information. High-resolution images, with their high pixel density, provide greater detail, such as anatomical, physiological, and functional metabolic information. Providing high-resolution images can greatly assist physicians in making accurate diagnoses and improve the performance of computer-assisted diagnosis.

[0003] Modern medical imaging relies on high-tech equipment—X-ray machines, CT (computed tomography), and MRI. While imaging quality improves with hardware upgrades, obtaining high-resolution images remains challenging due to practical limitations. For example, the simplest and most effective way to obtain high-resolution CT images is to increase the dose, but this results in higher radiation exposure. MRI, on the other hand, requires longer scan times, which can exponentially increase costs. Consequently, most hospitals generally avoid scanning at very high resolutions, as high-resolution images are often difficult to obtain. Conversely, with the increasing reliance on imaging for clinical diagnosis, hospitals have accumulated a large number of thick-slice images used for initial disease screening.

[0004] Therefore, mapping low-resolution images to high-resolution images is of great significance both for the precise diagnosis and treatment of new diseases and for the research of old image information mining.

[0005] Currently, there are two main methods to improve the resolution of medical images: physical methods and algorithmic methods.

[0006] Physical methods can mainly rely on increasing the number of sensors and increasing the MR magnetic field strength, but this may cause serious noise and greatly increase the imaging cost.

[0007] Algorithmic methods are mainly divided into three categories: interpolation-based methods, reconstruction-based methods, and learning-based methods. 1) Interpolation-based methods: mainly include nearest neighbor interpolation, bilinear interpolation, and cubic spline interpolation. Although simple and fast, they will cause the reconstructed image to be blurred and cannot introduce additional effective high-frequency information, and the effect of improving image resolution is limited; 2) Reconstruction-based methods: iterative back-projection method, convex set back-projection method, maximum a posteriori probability method, which converge relatively slowly, rely on the image degradation model, and are greatly affected by the quality of the original image; 3) Learning methods based on certain rules: sparse learning-based, dictionary-based, and neural network-based learning methods. The first two rely on a certain sample library, while deep learning based on neural networks can learn the features of the image in a supervised manner through the network, and then use it for image reconstruction. At present, this method is significantly better than the other methods.

[0008] Most neural network learning methods are based on natural images. They use a low-resolution image obtained by downsampling a high-resolution image as the network input. These images are then processed through multiple layers of convolution to supplement the image details and reconstruct a result similar to the high-resolution image. However, the images reconstructed using existing neural network learning methods have a consistent degree of restoration in every region of the image, and some regions may even have significantly poor restoration. This makes them ineffective for medical image restoration and reconstruction, failing to meet the requirements of medical diagnosis and scientific research. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to overcome the defect in the prior art that the method based on neural network learning cannot be effectively applied to the restoration and reconstruction of medical images, and to provide a training method for a super-resolution reconstruction network model, a scanning image processing method, an electronic device and a computer-readable medium.

[0010] The present invention solves the above technical problems through the following technical solutions:

[0011] According to one embodiment of the present invention, a method for training a super-resolution reconstruction network model is provided, comprising:

[0012] Acquire a low-resolution training image and a high-resolution training image corresponding to the low-resolution training image, wherein the pixel density of the high-resolution training image is higher than the pixel density of the corresponding low-resolution training image;

[0013] Inputting the low-resolution training image into a super-resolution reconstruction network model to be trained;

[0014] Performing image reconstruction processing on the low-resolution training image using the super-resolution reconstruction network model to obtain a super-resolution training image corresponding to the low-resolution training image, wherein a pixel density of the super-resolution training image is higher than or equal to a pixel density of the corresponding high-resolution training image;

[0015] Calculating a first loss between the super-resolution training image and the high-resolution training image;

[0016] Identifying an attention region from the super-resolution training image and identifying the same attention region from the high-resolution training image, and calculating a second loss between the super-resolution training image and the high-resolution training image in which the attention region is identified;

[0017] The super-resolution reconstruction network model is trained according to the first loss and the second loss.

[0018] Optionally, after the step of acquiring a low-resolution training image and a high-resolution training image corresponding to the low-resolution training image, the training method further comprises:

[0019] The low-resolution training images and high-resolution training images are registered to maintain image consistency.

[0020] Optionally, the steps of identifying an attention region from the super-resolution training image, identifying the same attention region from the high-resolution training image, and calculating a second loss between the super-resolution training image and the high-resolution training image in which the attention region is identified, specifically include:

[0021] Identifying an attention region from the super-resolution training image and identifying the same attention region from the high-resolution training image;

[0022] According to the distance between the pixel point of the training image and the identified attention area, a corresponding loss calculation weight is assigned to each pixel point of the training image, where the distance between the pixel point and the attention area and the loss calculation weight assigned to the pixel point are relative;

[0023] The second loss between the super-resolution training image and the high-resolution training image in which the attention area is identified is calculated by calculating the weight corresponding to each pixel.

[0024] Optionally, the training method further includes:

[0025] performing tissue segmentation on the super-resolution training image and the high-resolution training image respectively using the tissue structure diagram to obtain a tissue segmentation map of the super-resolution training image and a tissue segmentation map of the high-resolution training image;

[0026] calculating a third loss between the tissue segmentation map of the super-resolution training image and the tissue segmentation map of the high-resolution training image;

[0027] The super-resolution reconstruction network model is trained according to the first loss and the third loss.

[0028] Optionally, the super-resolution reconstruction network model includes several residual blocks;

[0029] Each of the residual blocks includes at least two first convolutional layers, an activation function, two second convolutional layers and a constant scaling layer;

[0030] The first convolutional layer includes a convolutional layer with a size of 3×1×1, and the second convolutional layer includes a convolutional layer with a size of 1×3×3.

[0031] Optionally, the super-resolution reconstruction network model includes any one or more of a dynamic neural network model, a convolutional neural network model, a fully convolutional network model, a deep neural network model, a generative adversarial network model, a recurrent neural network model, a deep residual network model and a long short-term memory network model.

[0032] According to another embodiment of the present invention, a scanned image processing method is provided, comprising:

[0033] Acquire a scanned image;

[0034] Inputting the scanned image into a super-resolution reconstruction network model trained by the super-resolution reconstruction network model training method as described above;

[0035] Performing image reconstruction processing on the scanned image through the super-resolution reconstruction network model;

[0036] Output the super-resolution reconstructed image after image reconstruction processing.

[0037] Optionally, after the step of outputting the super-resolution reconstructed image, the scanned image processing method further includes:

[0038] An image for the attention area is reconstructed from the super-resolution reconstructed image, and abnormality detection is performed on the image for the attention area to screen out abnormalities in the attention area.

[0039] According to another embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the super-resolution reconstruction network model or the scanning image processing method as described above is implemented.

[0040] According to another embodiment of the present invention, a computer-readable medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the training method of the super-resolution reconstruction network model or the scanning image processing method as described above is implemented.

[0041] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0042] The positive progress effect of the present invention is:

[0043] The present invention not only uses pairs of real low-resolution images and high-resolution images to train the network to output super-resolution images, but also adds attention areas. Different attention areas can be set according to different tissues of concern, and image details can be flexibly selected for restoration, which significantly improves the restoration effect of the image focus area. Therefore, it can be effectively applied to the restoration and reconstruction of medical images, meeting various requirements for medical diagnosis and scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The features and advantages of the present invention will be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

[0045] Figure 1 4 is a flow chart of a method for training a super-resolution reconstruction network model according to an embodiment of the present invention.

[0046] Figure 2 FIG. 4 is a flow chart of a scanned image processing method according to another embodiment of the present invention.

[0047] Figure 3a Schematic diagram of the process of training super-resolution reconstruction network model.

[0048] Figure 3b Schematic diagram of the structure of the super-resolution reconstruction network model.

[0049] Figure 3c Schematic diagram for calculating the loss between super-resolution training images and high-resolution training images.

[0050] Figure 4 A comparative display of image results obtained based on the super-resolution reconstruction network model.

[0051] Figure 5 FIG. 4 is a schematic diagram of a clinical diagnosis workflow according to another embodiment of the present invention.

[0052] Figure 6aSchematic diagram of conventional magnetic resonance imaging of a patient with left hippocampal sclerosis.

[0053] Figure 6b Schematic diagram of super-resolution reconstructed images of a patient with left hippocampal sclerosis.

[0054] Figure 6c Schematic diagram of a real high-resolution image of a patient with left hippocampal sclerosis.

[0055] Figure 7 Schematic diagram of the structure of an electronic device for implementing a training method for a super-resolution reconstruction network model or a scanned image processing method according to another embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0057] In order to overcome the above-mentioned defects that currently exist, this embodiment provides a training method for a super-resolution reconstruction network model, including: obtaining a low-resolution training image and a high-resolution training image corresponding to the low-resolution training image, wherein the pixel density of the high-resolution training image is higher than the pixel density of the corresponding low-resolution training image; inputting the low-resolution training image into the super-resolution reconstruction network model to be trained; performing image reconstruction processing on the low-resolution training image through the super-resolution reconstruction network model to obtain a super-resolution training image corresponding to the low-resolution training image, wherein the pixel density of the super-resolution training image is higher than or equal to the pixel density of the corresponding high-resolution training image; calculating a first loss between the super-resolution training image and the high-resolution training image; identifying an attention area from the super-resolution training image and identifying the same attention area from the high-resolution training image, and calculating a second loss between the super-resolution training image and the high-resolution training image in which the attention area is identified; and training the super-resolution reconstruction network model according to the first loss and the second loss.

[0058] Image resolution refers to the amount of information stored in an image, expressed as the number of pixels per inch (PI), commonly referred to as pixels per inch (PII). Generally speaking, higher image resolution means more detail and greater information content. Image resolution can be categorized into spatial and temporal resolutions. Typically, resolution is expressed as the number of pixels per dimension, such as in a 64x64 two-dimensional image. However, resolution is not synonymous with the number of pixels. For example, an image magnified five times through interpolation does not necessarily indicate increased detail. Image super-resolution reconstruction focuses on restoring lost detail, or high-frequency information, in an image. In many electronic imaging applications, high-resolution (HR) images are often desired. However, due to various factors, such as equipment and sensors, the resulting images are often low-resolution (LR). The most direct way to increase spatial resolution is to reduce pixel size through sensor manufacturing technology (for example, by increasing the number of pixels per unit area). Another approach to increasing spatial resolution is to increase chip size, thereby increasing image capacity. Because it is difficult to improve the coupling conversion rate of large capacities, this method is generally not considered effective, thus leading to the development of image super-resolution technology.

[0059] Super-resolution imaging refers to restoring a high-resolution image from a low-resolution image or image sequence. Image super-resolution technology is divided into super-resolution restoration and super-resolution reconstruction.

[0060] Super-resolution (SR) involves increasing the resolution of an original image through hardware or software methods. The process of obtaining a high-resolution image from a series of low-resolution images is known as super-resolution reconstruction. The core idea of ​​SR reconstruction is to trade temporal bandwidth (capturing a multi-frame sequence of images of the same scene) for spatial resolution, achieving the conversion of temporal resolution to spatial resolution.

[0061] In this embodiment, the super-resolution reconstruction network model may include any one or more of a dynamic neural network model (DMN), a convolutional neural network model (CNN), a fully convolutional network model (FCN), a deep neural network model (DNN), a generative adversarial network model (GAN), a recurrent neural network model (RNN), a deep residual network model (DRN) and a long short-term memory network model (LSTM), and can be selected and adjusted accordingly according to actual needs.

[0062] In this embodiment, not only are pairs of real low-resolution and high-resolution images used to train the network to output super-resolution images, but attention regions are also added. Different attention regions can be set according to different tissues of concern, and image details can be flexibly selected for restoration, which significantly improves the restoration effect of the image focus area. Therefore, it can be effectively applied to the restoration and reconstruction of medical images, meeting various requirements for medical diagnosis and scientific research.

[0063] Specifically, as an embodiment, Figure 1 As shown, the training method of the super-resolution reconstruction network model provided in this embodiment mainly includes the following steps:

[0064] Step 101: Obtain a low-resolution training image and a corresponding high-resolution training image.

[0065] In this step, as an optional implementation, a pair of real brain MR T1-weighted low-resolution images and high-resolution images are used as low-resolution training images and corresponding high-resolution training images.

[0066] Of course, the images are not limited to brain images, but can also be abdominal images, lung images or chest images, etc., and the images are not limited to MR images, but can also be CT images, DR (digital X-ray images) images or natural images, etc. This embodiment does not specifically limit the image type, and only needs to increase the required corresponding training data, and can make corresponding selections and adjustments according to actual needs.

[0067] In this step, after obtaining the low-resolution training image and the corresponding high-resolution training image, the low-resolution training image and the high-resolution training image are registered to maintain image consistency. Although this embodiment is not limited to directionality, as an optional implementation, the high-resolution training image can also be registered to the low-resolution training image to maintain image consistency.

[0068] As a preferred embodiment, the low-resolution training image and the high-resolution training image are linearly registered to maintain image consistency.

[0069] Specifically, all acquired brain MR images were preprocessed: first, the low-resolution original training images were converted from the intra-slice resolution of 0.75 × 0.75 mm to 2 , the layer thickness is about 4-5mm and resampled to 1×1×1mm 3 The standard low-resolution training image and the high-resolution training image are then linearly registered to maintain image consistency. The segmentation tool is then used to obtain a brain segmentation mask for the low-resolution training image, and the resulting mask is then used to remove the skull from both the low-resolution and high-resolution training images.

[0070] Step 102: Input the low-resolution training image into the super-resolution reconstruction network model to be trained.

[0071] As a preferred implementation, in this step summary, the super-resolution reconstruction network model is implemented using a dynamic neural network model (DMN), but is not limited thereto.

[0072] Step 103: Perform image reconstruction processing on the low-resolution training image using a super-resolution reconstruction network model to obtain a corresponding super-resolution training image.

[0073] In this step, refer to Figure 3a 、 Figure 3b and Figure 3c As shown in the figure, during training, the sparse representation-based super-resolution reconstruction (SCSR) technique uses the original low-resolution and high-resolution training images as DMN input. After the image preprocessing part (preprocessing) in the DMN, the HR image without the skull and registered with the low-resolution image domain is obtained. After the rest of the DMN network, the super-resolution training image (SR image) after super-reconstruction is obtained.

[0074] During testing, you only need to input new original low-resolution data into the trained DMN network to obtain a higher-resolution SR image.

[0075] Specific reference Figure 3b As shown, in this embodiment, the DMN body is generally composed of 32 residual blocks, each residual block includes: two first convolutional layers, an activation function (Relu), two second convolutional layers, and a constant scaling layer (scale), wherein the first convolutional layer includes a convolutional layer with a size of 3×1×1, and the second convolutional layer includes a convolutional layer with a size of 1×3×3.

[0076] For the super-resolution reconstruction network model, since it adopts a restoration strategy that directly supplements image information, the network input (low-resolution training images) and output (high-resolution training images) are of equal size. Similarly, no upsampling module is required in the network near the output layer. In each residual block, two consecutive convolutional layers (conx) of 3×1×1 (network size) and 1×3×3 are used to adapt to the anisotropy of data information. Since more information is already present within the layer, more information needs to be supplemented between layers. This reduces the computational complexity compared to using only a single 3×3×3 convolution.

[0077] Specifically, a 1×3×3 convolution extracts primary features. Next, 32 residual blocks allow the network to learn deep features from the high-resolution image. Finally, a 3×3×3 convolution is used to fuse these high-level features with the low-resolution image, supplementing the low-resolution image with the details doctors need. In each residual block, because the low-resolution data used in the experiment has varying resolutions within and across layers, the amount of high-level information required also varies. Therefore, continuous 3×1×1 and 1×3×3 convolutions are designed to adapt and reduce computational effort.

[0078] As an experimental example, 1740 pairs of real data were used, of which 1328 were used to train the model and the rest were used for testing.

[0079] Due to computational limitations, a crop operation was performed on 3D images. The network input path (channel) size was [32, 32, 32], and its grayscale value was normalized to [-1, 1]. Corresponding processing was also performed on high-resolution training images.

[0080] Step 104: Calculate a first loss between the super-resolution training image and the high-resolution training image.

[0081] In this step, refer to Figure 3a As shown, the first loss (Loss) is calculated using the obtained SR image and HR image, and the parameters of the DMN network are updated using the gradient backpropagation method.

[0082] Specifically, the grayscale images of the SR image and HR image obtained by DMN are used to calculate the first loss between the two:

[0083] MSSIM Loss:

[0084] (where x, y represent the pixels of the SR and HR images respectively, μ is the mean, σ is the variance, and C is a constant).

[0085] Step 105 : Determine whether the attention area can be identified from the training image. If so, execute step 106 ; if not, execute step 107 .

[0086] In this step, it is detected whether the training image specifies an attention area to be focused on. If so, step 106 is executed; if not, step 107 is executed.

[0087] Step 106: Calculate a second loss between the super-resolution training image and the high-resolution training image in which the attention region is identified. After executing step 106, execute step 109.

[0088] In this step, an attention region is identified from the super-resolution training image and the same attention region is identified from the high-resolution training image, and a second loss is calculated between the super-resolution training image and the high-resolution training image in which the attention region is identified.

[0089] As an optional implementation, this step may specifically include:

[0090] Identify attention regions from super-resolution training images and identify the same attention regions from high-resolution training images;

[0091] According to the distance between the pixel point of the training image and the identified attention area, a corresponding loss calculation weight is assigned to each pixel point of the training image, wherein the distance between the pixel point and the attention area and the loss calculation weight assigned to the pixel point are in a relative relationship. In this embodiment, the distance between the pixel point and the attention area and the loss calculation weight assigned to the pixel point can be in an inversely proportional relationship, that is, the smaller the distance between the pixel point and the attention area, the greater the loss calculation weight assigned to the pixel point, and conversely, the greater the distance between the pixel point and the attention area, the smaller the loss calculation weight assigned to the pixel point;

[0092] The second loss between the super-resolution training image and the high-resolution training image that identify the attention area is calculated by calculating the weight corresponding to the loss of each pixel.

[0093] Specifically, the grayscale image is used to calculate the weighted second loss between the SR image and the HR image:

[0094] MSE Loss: (i represents a pixel),

[0095] The weights here come from two parts: In the attention area (the hippocampus in this example), the weighting method for calculating DiceLoss is consistent; however, in order to better distinguish the tissue edges of white matter, gray matter, and cerebrospinal fluid, a tissue segmentation weight is added. Using the tissue segmentation result map, the distance from each point to its nearest tissue edge is calculated to obtain the tissue segmentation distance matrix. Weights are also assigned according to this matrix, and the rule is that the smaller the distance, the greater the weight. The two weight matrices are then combined to form the weight matrix of MSE Loss.

[0096] In this step, the above-mentioned attention area can be a predefined area, or it can be a key focus area selected using the tissue segmentation result map. Therefore, this embodiment does not specifically limit the type of the above-mentioned attention area and the setting or identification method, and can be selected and adjusted accordingly according to actual needs.

[0097] As an optional implementation, specifically, a pre-trained tissue segmentation tool is used to segment the MR training image to obtain a corresponding tissue segmentation result map, and a specific tissue area and several surrounding areas are selected from the obtained tissue segmentation result map as the focus area, thereby setting the attention area in the MR training image.

[0098] Step 107: Use the tissue structure diagram to perform tissue segmentation on the super-resolution training image and the high-resolution training image. After executing step 107, execute step 108.

[0099] In this step, tissue segmentation is performed on the super-resolution training image and the high-resolution training image using the tissue structure diagram to obtain a tissue segmentation map of the super-resolution training image and a tissue segmentation map of the high-resolution training image.

[0100] Step 108 : Calculate a third loss between the tissue segmentation maps of the super-resolution training image and the high-resolution training image. After executing step 108 , execute step 109 .

[0101] Specifically, the SR image and HR image are segmented using a pre-trained segmentation tool to obtain a tissue segmentation result map and a brain region segmentation result map of the HR image. The hippocampus and several surrounding areas are selected as the focus areas from the brain region segmentation result map.

[0102] The brain segmentation result map is used to calculate the relative distance between each position and the focus area to obtain a distance matrix, and then the segmentation map is used to calculate the third loss between the SR image and the HR image:

[0103] Dice Loss: (SR tissue is the tissue segmentation map of SR, HR tissue is the tissue segmentation map of HR, i, j represents pixel points), and weighted according to the obtained distance matrix - the closer to the area of ​​interest, the greater the weight.

[0104] Step 109: Train a super-resolution reconstruction network model according to the first loss, the second loss, and the third loss.

[0105] In this step, of course, the super-resolution reconstruction network model can also be trained according to the first loss and the second loss or according to the first loss and the third loss according to actual conditions.

[0106] Specifically, the loss L that is ultimately used to update the parameters of the training super-resolution reconstruction network model is the weighted sum of three losses:

[0107]

[0108] Among them, α, β, and γ are hyperparameters.

[0109] In this embodiment, the attention area can be varied in many ways according to the following two rules: 1) focusing on a single brain region, with only one brain region and the surrounding area as the attention area; 2) combining multiple brain regions to form n types of attention areas in any arrangement.

[0110] Figure 4 A set of low-resolution images, super-resolution images reconstructed by the network model trained by the training method provided in this embodiment, and high-resolution images acquired directly are shown. It can be seen that for the coronal image, the low-resolution image (network input) is very blurry due to the large layer thickness. The super-resolution image reconstructed by the network model looks very smooth to the human eye, but the details - such as the hippocampus - can also be effectively distinguished from the surrounding areas - the parahippocampal gyrus. The effect of the super-resolution image is almost comparable to that of the high-resolution image. The zoom-in part also illustrates this problem well.

[0111] In this embodiment, the super-resolution image obtained by the super-resolution reconstruction network model and the high-resolution image obtained by scanning at the cost of a very long time are expected to be as similar as possible. However, since the network pays equal attention to the entire image, the super-resolution image cannot meet the visual requirements in the relatively fine structure part. In order to better distinguish the edge structure of the tissue, this embodiment adds a tissue segmentation map to supervise the super-resolution network. Using a pre-trained tissue segmentation tool and fixing the parameters, this tool is used to obtain the tissue segmentation results of the super-resolution image and the high-resolution image. If the two images are similar enough, then their tissue segmentation maps should also be sufficiently similar. Adhering to this principle, the penalty is increased at the tissue boundary.

[0112] As the specific problems mentioned above require specific analysis, different diseases are often caused by different lesions. In this case, the areas of focus will be different. With a problem-oriented concept, the attention area is added. First, determine the part you want to focus on (the hippocampus and the area around the hippocampus); then use the brain area segmentation results obtained in the preprocessing part to generate the attention area mask. Before calculating the loss of the super-resolution image and the high-resolution image, first calculate the distance between each point and the attention area to obtain the distance matrix of the entire image with respect to the attention area. Different weights are assigned to each point on the image according to the size of the distance. The closer it is to the attention area, the greater the loss weight will be, the stronger the attention the network will give it, and the closer the reconstruction result will be to the high-resolution image.

[0113] As another example, Figure 2 As shown, the scanned image processing method provided in this embodiment mainly includes the following steps:

[0114] Step 201: Acquire a scanned image.

[0115] In this embodiment, the scanned image may be an MR image, a CT image, a DR image, or a natural image, etc. This embodiment does not specifically limit the type of scanned image, and corresponding selection and adjustment may be made according to actual needs.

[0116] Step 202: Input the scanned image into the trained super-resolution reconstruction network model.

[0117] In this step, the scanned image is input into the super-resolution reconstruction network model trained by the training method of the super-resolution reconstruction network model in the above embodiment.

[0118] Step 203: Perform image reconstruction processing on the scanned image using a super-resolution reconstruction network model to obtain a super-resolution reconstructed image.

[0119] In this step, a detail restoration process is performed on a preset attention area of ​​the scanned image to obtain a super-resolution reconstructed image with restored details of the attention area.

[0120] Step 204: reconstruct an image for the attention area from the super-resolution reconstructed image and perform anomaly detection to screen out anomalies in the attention area.

[0121] In this step, abnormality detection can be performed on the image of the attention area through artificial intelligence and other means to screen out abnormalities in the attention area, thereby facilitating medical diagnosis or scientific research.

[0122] The scanning image processing method provided in this embodiment can not only output super-resolution images, but also add attention areas. Different attention areas can be set according to different tissues of concern, and image details can be flexibly selected for restoration, which significantly improves the restoration effect of the image focus area. Therefore, it can be effectively applied to the restoration and reconstruction of medical images, meeting various requirements for medical diagnosis and scientific research.

[0123] This embodiment also provides a simple and convenient workflow for auxiliary diagnosis of clinical diseases based on image analysis. The workflow is implemented using the scanning image processing method of the above embodiment. The workflow can greatly reduce the time and cost required for patients to undergo imaging examinations, simplify the steps for detecting minor lesions; reduce the burden of scheduling films in the imaging department, maximize the utility of equipment resources; and change the doctor's previous diagnostic process of requiring repeated filming of suspected diseases.

[0124] Currently, the existing workflow for clinical disease-assisted diagnosis based on imaging analysis generally involves the patient verbally describing their condition, followed by a preliminary analysis and recommendation for an X-ray. The patient then proceeds to the imaging department for an X-ray according to the medical checklist, and upon receiving the results, they return to the doctor for confirmation of their condition. The radiologist then performs a routine MR scan on the patient. A routine MR scan, also known as a plain scan, typically consists of only a few two-dimensional sequences, such as T1WI, T2WI, and T2FLAIR. A two-dimensional sequence scan involves scanning one two-dimensional plane at a time, such as in a bottom-to-top transverse plane, and then moving upward, typically 5 mm, to scan the next plane.

[0125] Conventional MRI, or thick-slice images, can only screen for gross structures, such as excluding large tumors and space-occupying lesions. Details in other planes, such as the coronal and sagittal planes, are blurred, and only cross-sectional images are relatively clear. Lesions smaller than the slice thickness (5mm in this example) cannot be visualized on conventional MRI, and minute lesions in fine structures are overlooked. This makes it particularly inadequate for screening for structural diseases such as the hippocampus, which, despite its small size, occupies a crucial position in the human body.

[0126] After excluding large structural disease, experienced physicians typically request that patients (those with significant clinical manifestations and normal routine plain scan images, who may have a neurodegenerative disease to some extent) undergo a high-resolution MR scan to confirm the disease. This process further increases patient time and cost, while also placing an additional burden on hospital equipment operations.

[0127] To address the above-mentioned existing deficiencies, the workflow for clinical disease auxiliary diagnosis based on image analysis provided by this embodiment has the following main effects:

[0128] 1) This method is based on routine clinical scan images and uses the scan image processing method mentioned above to perform super-resolution image reconstruction to obtain images with more high-frequency information.

[0129] 2) Clinicians analyze and diagnose diseases based on the reconstructed super-resolution images.

[0130] 3) Streamlined the process of multiple clinical diagnoses and feedback.

[0131] 4) There is no need to repeatedly perform high-resolution image scans to confirm the precise location of the lesion.

[0132] 5) Generally speaking, patients with multiple lesions do not need to undergo multiple high-resolution image scans.

[0133] 6) You can choose whether to add AI (artificial intelligence) multi-case sample diagnosis results, combining artificial intelligence with human intelligence to reduce the possibility of misdiagnosis due to physiological fatigue.

[0134] As an optional implementation, the diagnosis of hippocampal sclerosis is taken as an example to illustrate the clinical auxiliary diagnosis workflow described above. Of course, this workflow can also be effectively applied to diseases that require analysis of subtle changes in images, including but not limited to: early diagnosis of tumors, diagnosis and treatment of Alzheimer's disease, localization of microbleeding lesions, etc.

[0135] The hippocampus is a sensitive area that is easily damaged. Trauma, ischemia and hypoxia, inflammatory response and degeneration can all cause hippocampal lesions. Hippocampal sclerosis was first proposed by Falcomer and others. It is also called mesial temporal sclerosis. It is the most common pathological type of intractable temporal lobe epilepsy and is causally related to epilepsy. Its cause may be related to various injuries in infancy and childhood (trauma, convulsions, febrile spasms, etc.). The hippocampal structure includes the hippocampus, subiculum, dentate gyrus, adjacent entorhinal cortex and hippocampal remnants of the surrounding corpus callosum. Common MRI manifestations are: hippocampal atrophy (the most common and reliable indication); disappearance of the superficial sulcus of the hippocampal head; disappearance of the internal anatomical structure of the hippocampus; temporal lobe atrophy; and enlargement of the temporal horn of the lateral ventricle on the affected side.

[0136] refer to Figure 5 As shown, the clinical workflow of this embodiment for diagnosing patients with hippocampal sclerosis is as follows: the patient exhibits certain clinical signs—automatics such as groping and lip smacking—and promptly seeks medical treatment at the hospital; the doctor makes a preliminary diagnosis of the symptoms described by the patient and then recommends that the patient use imaging methods for confirmation; the doctor issues an examination order—a routine clinical MRI scan; after reconstruction using the super-resolution reconstruction network model as described above, a super-resolution medical image that can be used for auxiliary diagnosis is obtained; at this time, an experienced doctor can give a diagnosis based on the subtle lesions and clinical symptoms in the fine image and formulate a treatment plan.

[0137] As an optional solution, AI that has learned countless cases can also provide imaging marker detection and attach relevant case detection results, quantitative features and other information. Doctors refer to the results given by AI and combine their own experience to give a final diagnosis. For example, by using clinical thick-layer images as input, after automatic image super-resolution reconstruction, the AI ​​algorithm can finely segment the hippocampus area and obtain quantitative features. By feeding these quantitative features into the classifier through omics methods, it is possible to automatically classify patients with left hippocampal sclerosis, patients with right hippocampal sclerosis and normal controls into three categories, thereby automatically prompting doctors that this patient may have hippocampal sclerosis and providing unilateral information to facilitate doctors to find the problem and make a diagnosis. As an experimental example, experiments were conducted on 182 cases, and the results showed that the classification accuracy of patients with left hippocampus was 92.3%, and the classification accuracy of patients with right hippocampus was 94.0%.

[0138] After using the above workflow, the filming process will be greatly simplified. Figure 6b and Figure 6c As shown in the figure, the visual effect of the super-resolution reconstructed image is very close to the high-resolution image directly collected, which is time-consuming and laborious, and the segmentation effect of the hippocampus is basically consistent.

[0139] contrast Figure 6a 、 Figure 6b and Figure 6c It can be seen that the existing clinical routine MR plain scan images have completely blurred the hippocampus area in the coronal and sagittal planes, and the boundaries of various tissues are also difficult to determine. Only in the cross section can the GM / WM / CSF (gray matter / white matter / cerebrospinal fluid) be observed more clearly, and only diseases with a lesion range greater than 5mm can be displayed across layers in the axial image. However, compared with super-resolution images and high-resolution images, the visual effect is still slightly inferior. Since the distance between layers is relatively large during scanning, some effective information is lost, which will also affect the reconstruction results of the intra-layer images. In this case, using the new workflow provided by this embodiment and directly using the reconstructed super-resolution images for disease diagnosis is undoubtedly an economical and convincing choice.

[0140] Figure 7 The figure is a schematic diagram of the structure of an electronic device according to another embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the super-resolution reconstruction network model training method or scanned image processing method described in the above embodiments. Figure 7 The electronic device 30 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0141] like Figure 7As shown, the electronic device 30 may be a general-purpose computing device, such as a server device. Components of the electronic device 30 may include, but are not limited to, the at least one processor 31, the at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0142] The bus 33 includes a data bus, an address bus, and a control bus.

[0143] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0144] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0145] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the training method of the super-resolution reconstruction network model or the scanned image processing method in the above embodiments of the present invention.

[0146] The electronic device 30 may also communicate with one or more external devices 34 (e.g., a keyboard, a pointing device, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 7 As shown, the network adapter 36 communicates with the other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in conjunction with the model-generated device 30, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0147] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above may be embodied in a single unit / module. Conversely, the features and functions of a single unit / module described above may be further divided and embodied by multiple units / modules.

[0148] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the training method of the super-resolution reconstruction network model or the scanning image processing method in the above embodiment are implemented.

[0149] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0150] In a possible embodiment, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the training method or scanning image processing method of the super-resolution reconstruction network model in the above embodiment.

[0151] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0152] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, and such changes and modifications are intended to fall within the scope of the present invention.

Claims

1. A training method for a super-resolution reconstruction network model, characterized in that: include: Acquire a low-resolution training image and a high-resolution training image corresponding to the low-resolution training image, wherein the high-resolution training image has a higher pixel density than the corresponding low-resolution training image; wherein the image is a medical image; Inputting the low-resolution training image into a super-resolution reconstruction network model to be trained; Performing image reconstruction processing on the low-resolution training image using the super-resolution reconstruction network model to obtain a super-resolution training image corresponding to the low-resolution training image, wherein a pixel density of the super-resolution training image is higher than or equal to a pixel density of the corresponding high-resolution training image; Calculating a first loss between the super-resolution training image and the high-resolution training image; performing tissue segmentation on the super-resolution training image and the high-resolution training image respectively using the tissue structure diagram to obtain a tissue segmentation map of the super-resolution training image and a tissue segmentation map of the high-resolution training image; Identifying an attention region from the tissue segmentation map of the super-resolution training image and identifying the same attention region from the tissue segmentation map of the high-resolution training image, assigning a corresponding loss calculation weight to each pixel of the training image based on the distance between the pixel of the training image and the identified attention region, wherein the distance between the pixel and the attention region and the loss calculation weight assigned to the pixel are in a relative relationship; calculating a second loss between the super-resolution training image and the high-resolution training image using the loss calculation weight corresponding to each pixel; calculating a third loss between the tissue segmentation map of the super-resolution training image and the tissue segmentation map of the high-resolution training image; The super-resolution reconstruction network model is trained according to the first loss, the second loss, and the third loss.

2. The training method according to claim 1, wherein: After the step of acquiring the low-resolution training image and the high-resolution training image corresponding to the low-resolution training image, the training method further comprises: The low-resolution training images and high-resolution training images are registered to maintain image consistency.

3. The training method according to any one of claims 1 to 2, characterized in that: The super-resolution reconstruction network model includes several residual blocks; Each of the residual blocks includes at least two first convolutional layers, an activation function, two second convolutional layers and a constant scaling layer; The first convolutional layer includes a convolutional layer with a size of 3×1×1, and the second convolutional layer includes a convolutional layer with a size of 1×3×3.

4. The training method according to claim 1, wherein: The super-resolution reconstruction network model includes any one or more of a dynamic neural network model, a convolutional neural network model, a fully convolutional network model, a deep neural network model, a generative adversarial network model, a recurrent neural network model, a deep residual network model and a long short-term memory network model.

5. A scanned image processing method, characterized in that: include: Acquire a scanned image; Inputting the scanned image into a super-resolution reconstruction network model trained by the super-resolution reconstruction network model training method according to any one of claims 1 to 4; Performing image reconstruction processing on the scanned image through the super-resolution reconstruction network model; Output the super-resolution reconstructed image after image reconstruction processing.

6. The scanned image processing method according to claim 5, wherein: After the step of outputting the super-resolution reconstructed image, the scanned image processing method further includes: An image for the attention area is reconstructed from the super-resolution reconstructed image, and abnormality detection is performed on the image for the attention area to screen out abnormalities in the attention area.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the training method of the super-resolution reconstruction network model as described in any one of claims 1 to 4 or the scanned image processing method as described in any one of claims 5 to 6.

8. A computer-readable medium having computer instructions stored thereon, characterized in that: When executed by a processor, the computer instructions implement the training method of the super-resolution reconstruction network model as described in any one of claims 1 to 4 or the scanned image processing method as described in any one of claims 5 to 6.

Images