Convergence degree prediction method for image reconstruction, image reconstruction method, device and equipment

Abstract

The invention relates to the technical field of image processing, and provides an image reconstruction convergence degree prediction method, an image reconstruction method, an image reconstruction device and electronic equipment. The method comprises the following steps: acquiring a to-be-predicted image generated in an iterative reconstruction mode; inputting the first feature data of the to-be-predicted image into the trained convergence degree prediction model, and outputting the convergence degree of the to-be-predicted image through the convergence degree prediction model; wherein the convergence degree is used for representing the feature difference between the to-be-predicted image and the corresponding target image, and the convergence degree prediction model is a deep learning model used for predicting the image reconstruction convergence degree. By adopting the method, the convergence degree of image reconstruction can be quantified, so that a reference is provided for accurate diagnosis of a focus.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a convergence degree prediction method for image reconstruction, an image reconstruction method, an image reconstruction device and an electronic device. BACKGROUND

[0002] In the field of medical imaging, in the iterative process of PET / SPECT reconstructed images, image reconstruction algorithms such as OSEM and regularization are usually used to generate images with less noise that can be used for clinical diagnosis through multiple iterative reconstruction. However, the pixel values in some areas of the reconstructed image often do not converge to their true values, that is, there is a certain feature difference between the reconstructed image and the corresponding target image, which will interfere with the diagnosis of the lesion. Therefore, how to quantify the convergence degree of image reconstruction to provide a reference for accurate diagnosis of the lesion becomes a technical problem that needs to be considered by those skilled in the art. SUMMARY

[0003] Therefore, the embodiments of the present application provide a convergence degree prediction method for image reconstruction, an image reconstruction method, an image reconstruction device and an electronic device, which can quantify the convergence degree of image reconstruction.

[0004] The first aspect of the embodiments of the present application provides a convergence degree prediction method for image reconstruction, comprising:

[0005] obtaining a to-be-predicted image generated by an iterative reconstruction method;

[0006] inputting first feature data of the to-be-predicted image into a trained convergence degree prediction model, and outputting a convergence degree of the to-be-predicted image through the convergence degree prediction model; wherein the convergence degree is used to represent a feature difference between the to-be-predicted image and a corresponding target image, and the convergence degree prediction model is a deep learning model used to predict the convergence degree of image reconstruction.

[0007] The embodiments of the present application pre-train a deep learning model used to predict the convergence degree of image reconstruction as a convergence degree prediction model. After obtaining the to-be-predicted image generated by the iterative reconstruction method, only the feature data of the to-be-predicted image needs to be input into the convergence degree prediction model for processing, and then the convergence degree of the to-be-predicted image can be output through the convergence degree prediction model. The convergence degree is used to represent a feature difference between the to-be-predicted image and a corresponding target image. Since the convergence degree prediction model outputs a quantitative numerical prediction result, the quantitative calculation of the convergence degree of image reconstruction is realized.

[0008] In an implementation manner of the embodiments of the present application, the first feature data is obtained by the following manner:

[0009] obtaining image information of the to-be-predicted image;

[0010] The image information of the image to be predicted is encoded to obtain the first feature data.

[0011] In another implementation of this application embodiment, the first feature data is obtained in the following way:

[0012] Obtain image information of the image to be predicted;

[0013] The image information of the image to be predicted is encoded to obtain the first feature vector;

[0014] Extract the image features of the image to be predicted to obtain the second feature vector;

[0015] The first feature vector and the second feature vector are fused to obtain the first feature data.

[0016] In one implementation of this application, the training process of the convergence prediction model includes:

[0017] Obtain a one-to-one corresponding sample image and a reference image; wherein the sample image and the reference image are generated using the same iterative reconstruction algorithm, the reference image has a higher number of iterations than the sample image, and the reference image corresponds to the highest convergence;

[0018] Based on the sample images and the benchmark images, we construct the training set data and the loss function;

[0019] A convergence prediction model is trained based on the training set data and by minimizing the loss function.

[0020] In one implementation of this application, training set data is constructed based on sample images and reference images, including:

[0021] Based on the image information and image features of the sample image, generate the second feature data of the sample image;

[0022] Based on the image information and image features of the reference image, generate the third feature data of the reference image;

[0023] The training set data is constructed by taking the second feature data as input and the difference between the second feature data and the third feature data as the output target.

[0024] In one implementation of this application, generating second feature data of the sample image based on the image information and image features of the sample image includes:

[0025] Perform region segmentation processing on the sample image to obtain the first region of interest in the sample image;

[0026] Extract the image features of the first region of interest and its neighboring regions to obtain the third feature vector;

[0027] The image information of the sample image is encoded to obtain the fourth feature vector;

[0028] The third feature vector is fused with the fourth feature vector to obtain the second feature data;

[0029] Based on the image information and image features of the reference image, third feature data of the reference image is generated, including:

[0030] Perform region segmentation processing on the reference image to obtain the second region of interest in the reference image;

[0031] The image features of the second region of interest and its adjacent regions are extracted to obtain the fifth feature vector;

[0032] The image information of the reference image is encoded to obtain the sixth feature vector;

[0033] The fifth feature vector is fused with the sixth feature vector to obtain the third feature data.

[0034] In one implementation of this application, after training the convergence prediction model, the method further includes:

[0035] Obtain multiple test images from the sample images;

[0036] Multiple test images are input into the convergence prediction model, and the convergence prediction model outputs the convergence prediction results of the multiple test images.

[0037] Based on the fourth feature data of multiple test images, the fifth feature data of the corresponding benchmark images, and the convergence prediction results of multiple test images, the confidence interval of the prediction results output by the convergence prediction model is calculated.

[0038] A second aspect of this application provides an image reconstruction method, including:

[0039] Obtain the image to be predicted generated through iterative reconstruction;

[0040] The feature data of the image to be predicted is input into the trained convergence prediction model, and the convergence of the image to be predicted is output by the convergence prediction model. The convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image. The convergence prediction model is a deep learning model used to predict the convergence of image reconstruction.

[0041] If the convergence meets the set conditions, the image to be predicted will be determined as the target image after reconstruction.

[0042] If the convergence does not meet the set conditions, the image to be predicted will continue to be reconstructed iteratively, and then the process will return to the step of inputting the feature data of the image to be predicted into the trained convergence prediction model.

[0043] A third aspect of this application provides an image reconstruction convergence prediction apparatus, comprising:

[0044] The image acquisition module is used to acquire the image to be predicted generated through iterative reconstruction.

[0045] The convergence prediction module is used to input the first feature data of the image to be predicted into the trained convergence prediction model, and output the convergence of the image to be predicted through the convergence prediction model; wherein, the convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image, and the convergence prediction model is a deep learning model used to predict the convergence of image reconstruction.

[0046] A fourth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the convergence prediction method for image reconstruction as provided in the first aspect of this application, or implements the image reconstruction method as provided in the second aspect of this application.

[0047] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the convergence prediction method for image reconstruction as provided in the first aspect of this application, or implements the image reconstruction method as provided in the second aspect of this application.

[0048] A sixth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the convergence prediction method for image reconstruction as provided in the first aspect of this application, or to execute the image reconstruction method as provided in the second aspect of this application.

[0049] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a flowchart of an image reconstruction convergence prediction method provided in an embodiment of this application;

[0051] Figure 2 This is a schematic diagram illustrating the training process of a convergence prediction model provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram of the prediction process of a convergence prediction model provided in an embodiment of this application;

[0053] Figure 4 This is a flowchart of an image reconstruction method provided in an embodiment of this application;

[0054] Figure 5 This is a schematic diagram of the structure of an image reconstruction convergence prediction device provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0057] In the field of medical image processing, various image reconstruction algorithms such as OSEM and regularization are commonly used to obtain low-noise images suitable for clinical diagnosis within a limited number of iterations. However, the pixel values ​​in some regions of the obtained images often do not converge to their true values, and the quantitative accuracy of image regions directly affects the diagnostic results of lesions. Therefore, how to quantify the convergence degree of image reconstruction to provide a reference for the accurate diagnosis of lesions has become a technical problem that needs to be considered by those skilled in the art.

[0058] To address the aforementioned technical problems, embodiments of this application provide a convergence prediction method, an image reconstruction method, an apparatus, and an electronic device for image reconstruction. By training a deep learning network, the convergence degree of image reconstruction is quantitatively calculated. For more specific technical implementation details of these embodiments, please refer to the method embodiments described below.

[0059] It should be understood that the execution subject of the various method embodiments of this application can be various types of electronic devices, such as desktop computers, tablet computers, mobile phones, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, netbooks, personal digital assistants (PDAs), medical diagnostic devices, etc. The embodiments of this application do not impose any restrictions on the specific type of electronic device.

[0060] Please see Figure 1 This paper illustrates an image reconstruction convergence prediction method provided by an embodiment of this application, comprising:

[0061] 101. Obtain the image to be predicted generated through iterative reconstruction;

[0062] First, the image to be predicted is obtained. The image to be predicted can be an image generated through iterative reconstruction using any image reconstruction algorithm. For example, the image to be predicted can be a clinical image obtained after 2-5 iterations of reconstruction using the OSEM algorithm. Its pixel features differ to some extent from those of the corresponding target image. The convergence prediction method provided in this application can be used to predict the quantified convergence, thereby enabling the assessment of the degree of feature difference between the image to be predicted and the corresponding target image based on the convergence.

[0063] 102. Input the first feature data of the image to be predicted into the trained convergence prediction model, and output the convergence of the image to be predicted through the convergence prediction model.

[0064] In this embodiment, a deep learning network is pre-trained as a convergence prediction model. The type and structure of the deep learning network used are not limited. After obtaining the image to be predicted, its feature data is input into the convergence prediction model for processing. The convergence prediction model can output the convergence of the image to be predicted. The feature data of the image to be predicted is denoted as the first feature data, which may include image features and information features of the image to be predicted. The training method of the convergence prediction model is described below.

[0065] In one implementation of this application, the training process of the convergence prediction model includes:

[0066] (1) Obtain a one-to-one corresponding sample image and reference image; wherein, the sample image and the reference image are generated using the same iterative reconstruction algorithm, the reference image has a higher number of iterations than the sample image, and the reference image corresponds to the highest convergence.

[0067] (2) Construct training set data and loss function based on sample images and benchmark images;

[0068] (3) Based on the training set data and by minimizing the loss function, a convergence prediction model is trained.

[0069] When constructing the training set, it is necessary to obtain one-to-one corresponding sample images and baseline images. Multiple sample and baseline images are obtained, and both are generated using the same iterative reconstruction algorithm. The baseline image undergoes a higher number of iterations than the sample images, and corresponds to the highest convergence. In practice, a sufficient number of original images can be collected. For each original image, a certain image reconstruction algorithm is used to reconstruct a sample image with fewer iterations. The same image reconstruction algorithm is then used to reconstruct a baseline image with more iterations, thus obtaining multiple one-to-one corresponding sample and baseline images. Since the baseline image has a high number of iterations (e.g., >50), the pixel mean or individual pixel values ​​of its image region can be considered to have completely converged to the corresponding real image; therefore, its convergence can be defined as the highest, 100%. As an example, the sample image can be an image reconstructed using a clinical protocol, such as an image reconstructed using the OSEM algorithm (20 subsets, 2 iterations); the baseline image can be an image reconstructed through multiple iterations, such as an image reconstructed using the OSEM algorithm (20 subsets, 50 iterations). After obtaining a sufficient number of sample images and benchmark images, the training set data and loss function for model training can be constructed based on these sample images and benchmark images. Then, based on the training set data and by minimizing the loss function, a convergence prediction model can be trained.

[0070] In one implementation of this application, training set data is constructed based on sample images and reference images, including:

[0071] (1) Generate the second feature data of the sample image based on the image information and image features of the sample image;

[0072] (2) Generate the third feature data of the reference image based on the image information and image features of the reference image;

[0073] (3) Using the second feature data as input and the difference between the second feature data and the third feature data as output target, the training set data is constructed.

[0074] When constructing the training set data based on sample and reference images, feature data is first extracted from both images. The feature data from the sample images is denoted as the second feature data, and the feature data from the reference images is denoted as the third feature data. When generating the feature data, the image information and image features of the corresponding images can be combined. That is, the second feature data is generated based on the image information and image features of the sample images, and vice versa. The image information can include the type of reconstruction algorithm used, the number of iterations, the number of subsets, filtering parameters, and regularization parameters. The image features can be the pixel values ​​at various locations in the image. In practice, both the image information and image features can be converted into corresponding feature vectors, and then the two feature vectors are fused to obtain the corresponding feature data. Finally, using the second feature data as input and the difference between the second and third feature data as the output target, the training set data is constructed. The difference between the second and third feature data can be defined as the convergence of the sample image. Specifically, it can be the feature ratio of a region in the sample image (e.g., a lesion region) to the corresponding region in the reference image. For example, it could be the average of the ratios of pixel values ​​at various locations within the region, the ratio of the average pixel values ​​at various locations within the region, the ratio of the maximum pixel value within the region, or the ratio of the minimum pixel value within the region, and so on. The feature ratio can be represented by k. When k = 100%, the convergence of the sample image is 100%; when k = 70%, the convergence of the sample image is 70%, and so on.

[0075] In one implementation of this application, generating second feature data of the sample image based on the image information and image features of the sample image includes:

[0076] (1) Perform region segmentation processing on the sample image to obtain the first region of interest of the sample image;

[0077] (2) Extract the image features of the first region of interest and its adjacent regions to obtain the third feature vector;

[0078] (3) Encode the image information of the sample image to obtain the fourth feature vector;

[0079] (4) The third feature vector and the fourth feature vector are fused to obtain the second feature data.

[0080] Similarly, based on the image information and image features of the reference image, third feature data of the reference image is generated, including:

[0081] (1) Perform region segmentation processing on the reference image to obtain the second region of interest of the reference image;

[0082] (2) Extract the image features of the second region of interest and its adjacent regions to obtain the fifth feature vector;

[0083] (3) Encode the image information of the reference image to obtain the sixth feature vector;

[0084] (4) The fifth feature vector and the sixth feature vector are fused to obtain the third feature data.

[0085] In many cases, users are only interested in a portion of an image, not the entire image. To reduce the computational burden of feature data processing, image segmentation can be performed first to obtain regions of interest (ROIs). The region extracted from the sample image after segmentation is denoted as the first ROI, and the region extracted from the baseline image is denoted as the second ROI. Typically, the ROI can be a lesion or organ, with a size exceeding one pixel. Extracting the ROI in this case reduces the computational burden of feature data processing. However, in special cases, the ROI may be equal to one pixel. In these cases, the meaning of the region is not considered, and the deep learning network performs point-to-point mapping, achieving the same effect as no image segmentation. As an example, for clinical images, PET images, CT images, and other anatomical information can be used with a deep learning network such as U-Net to extract and segment lesion areas as ROIs. After segmenting the ROI, image features of the ROI and its adjacent regions are extracted to obtain corresponding feature vectors. These feature vectors are then fused with feature vectors obtained after encoding the image information to obtain the corresponding feature data. Specifically, for the sample image, the feature vector obtained by extracting image features from the first region of interest and its neighboring regions is denoted as the third feature vector. The image information of the sample image is encoded, for example, by using embedding encoding to convert the image information into a low-dimensional vector, thus obtaining the fourth feature vector. Then, the third and fourth feature vectors are fused. This fusion can be achieved through vector multiplication, vector addition, or weighted summation. The result of vector fusion is used as the feature data of the sample image, i.e., the second feature data. For the reference image, the feature vector obtained by extracting image features from the second region of interest and its neighboring regions is denoted as the fifth feature vector. The image information of the reference image is encoded to obtain the sixth feature vector. Then, the fifth and sixth feature vectors are fused. The result of vector fusion is used as the feature data of the sample image, i.e., the third feature data. Finally, using the second feature data as input and the difference between the second and third feature data as the output target, the training set data can be constructed. Assuming the sample images are clinical images and the benchmark image is the gold standard image that can be considered a real image, as an example, the input of the training set data is the feature data of the lesions and their adjacent regions in the clinical image, and the output target is the ratio of the feature data of the lesions and their adjacent regions in the clinical image to the feature data of the lesions and their adjacent regions in the gold standard image; as another example, the input of the training set data is the feature data of the clinical image, and the output target is the ratio of the feature data of the clinical image to the feature data of the gold standard image.

[0086] As an example, assuming the region of interest is a lesion, the loss function constructed during model training can include the following three types:

[0087] (A) The difference between the output of the network in each iteration and the output target k for each pixel;

[0088] (B) The difference between the output result of each iteration of the network and the output target k at each pixel point in the lesion area;

[0089] (C) The difference between the output result of each iteration of the network and the average pixel value of the output target k in the lesion area.

[0090] The constructed loss function can be of any type, denoted by Loss = F(k, output). Taking L1 loss as an example, for type (A), its loss function can be expressed as:

[0091]

[0092] For type (B), its loss function can be expressed as:

[0093]

[0094] For type (C), its loss function can be expressed as:

[0095]

[0096] Where N represents the number of pixels in the image, n l The number of lesions is represented by n, the number of pixels within the lesion region is represented by i, the lesion index is represented by j, and the feature ratio between the sample image and the baseline image is represented by k.

[0097] After constructing the loss function, the model parameters are iteratively optimized based on the training set data and by minimizing the loss function, and finally a convergence prediction model is trained.

[0098] In one implementation of this application, after training the convergence prediction model, the method further includes:

[0099] (1) Obtain multiple test images from the sample images;

[0100] (2) Input multiple test images into the convergence prediction model, and output the convergence prediction results of multiple test images through the convergence prediction model;

[0101] (3) Based on the fourth feature data of multiple test images, the fifth feature data of the reference image corresponding to the multiple test images, and the convergence prediction results of multiple test images, the confidence interval of the prediction results output by the convergence prediction model is calculated.

[0102] To facilitate user evaluation of the convergence prediction model's accuracy, multiple test images can be obtained from the sample images as a test set. These test images are then input into the convergence prediction model for processing. The model outputs the convergence prediction results for these test images and calculates the confidence interval of the model's prediction results by comparing image feature data. Specifically, feature data from the multiple test images is obtained, denoted as the fourth feature data, and feature data from the corresponding baseline image is obtained, denoted as the fifth feature data. Based on the fourth feature data, some index parameters of the region of interest are calculated, such as the average pixel value, maximum pixel value, minimum pixel value, and pixel value at each location within the lesion area. Similarly, the same index parameters can be calculated based on the fifth feature data. By comparing these index parameters, a confidence interval of a certain level (e.g., 95%) for the convergence prediction can be obtained. Furthermore, images can be classified according to lesion type, and multiple test images of the same lesion type can be obtained to calculate the confidence interval for the convergence prediction under the current lesion type. For example, 100 images of liver lesions can be obtained as test images. These test images can be input into a convergence prediction model for processing to obtain 100 convergence prediction results. Based on the sample statistics of these 100 convergence prediction results, the confidence interval of the convergence prediction result corresponding to the liver lesion image can be calculated.

[0103] As an example, such as Figure 2 The diagram illustrates the training process of a convergence prediction model provided in this embodiment. After the model training begins, a large amount of raw data (i.e., original images) without iterative reconstruction is acquired. The same image reconstruction algorithm is used to perform image reconstruction operations on the original images with different numbers of iterations. Image reconstruction operations with higher iteration counts generate gold standard images, while those with lower iteration counts generate clinical images. Then, the image information of the clinical images and gold standard images is read, encoded, and image region segmentation is performed on the clinical images and gold standard images to obtain lesion regions. Training set data is constructed based on the feature data of the lesion regions and their adjacent regions, and a loss function is also constructed. Next, based on the training set data and by minimizing the loss function, the convergence prediction model is trained, continuously optimizing the training model parameters. Additionally, a portion of the clinical images can be selected for case testing, or indicator parameters of the lesion region can be calculated from the target image corresponding to the image to be predicted. These indicator parameters may include lesion size, lesion region pixel values, etc. By statistically analyzing the convergence prediction results of these clinical images, the confidence interval of the convergence prediction results can be calculated to determine the accuracy of the trained model.

[0104] After training the convergence prediction model according to the method described above, the first feature data of the image to be predicted is input into the trained convergence prediction model for processing, and the convergence of the image to be predicted can be obtained. This is a quantitative numerical prediction result.

[0105] In one implementation of this application embodiment, the first feature data is obtained in the following manner:

[0106] (1) Obtain image information of the image to be predicted;

[0107] (2) Encode the image information of the image to be predicted to obtain the first feature data.

[0108] Image information such as the reconstruction algorithm type, iteration count, subset size, filtering parameters, and regularization parameters of the image to be predicted is obtained. This image information is then encoded using methods such as embedding to obtain the corresponding feature vector, which serves as the first feature data. In this implementation, only the features of the image information are input to the convergence prediction model. The convergence prediction model compares and calculates based on the input image information features, outputting the convergence prediction result for the image to be predicted. Additionally, the confidence interval of the prediction result can also be output.

[0109] In another implementation of this application embodiment, the first feature data is obtained in the following way:

[0110] (1) Obtain image information of the image to be predicted;

[0111] (2) Encode the image information of the image to be predicted to obtain the first feature vector;

[0112] (3) Extract the image features of the image to be predicted to obtain the second feature vector;

[0113] (4) The first feature vector and the second feature vector are fused to obtain the first feature data.

[0114] On one hand, image information of the image to be predicted is encoded using methods such as embedding to obtain corresponding feature vectors, denoted as the first feature vector. On the other hand, image features of the image to be predicted are extracted, specifically image features of the region of interest (e.g., lesion region) contained in the image, such as lesion contour and size, to obtain the second feature vector. Then, the first and second feature vectors are fused. This fusion can be achieved through vector multiplication, vector addition, or weighted vector summation. The result after vector fusion is used as the first feature data. In this implementation, the convergence prediction model is input with fused features containing both image information features and image features. The convergence prediction model compares and calculates based on the input fused features, outputting the convergence prediction result of the image to be predicted, and can also output the confidence interval of the prediction result.

[0115] As an example, such as Figure 3 The diagram shown illustrates the prediction process of a convergence prediction model provided in this application embodiment. Once the model prediction process begins, the clinical image to be predicted is acquired, and the image information of the clinical image is read. The clinical image and its image information are input into the convergence prediction model for processing. The model performs feature data comparison and calculation, and finally outputs the convergence prediction result of the clinical image, as well as the confidence interval of the prediction result.

[0116] In this embodiment, a deep learning model for predicting the convergence of image reconstruction is pre-trained as a convergence prediction model. After obtaining the image to be predicted generated through iterative reconstruction, the feature data of the image to be predicted is input into the convergence prediction model for processing. The convergence prediction model then outputs the convergence of the image to be predicted, which characterizes the feature difference between the image to be predicted and the corresponding target image. Since the convergence prediction model outputs a quantified numerical prediction result, the quantification of the convergence degree of image reconstruction is achieved.

[0117] This application also proposes an image reconstruction method based on the above-described convergence prediction method. Please refer to [link to relevant documentation]. Figure 4 The image reconstruction method includes:

[0118] 401. Obtain the image to be predicted generated through iterative reconstruction;

[0119] 402. Input the feature data of the image to be predicted into the trained convergence prediction model, and output the convergence of the image to be predicted through the convergence prediction model.

[0120] For detailed explanations of steps 401-402, please refer to the previous description of obtaining the convergence of the image to be predicted using the convergence prediction model; these will not be repeated here.

[0121] 403. Determine whether the convergence meets the set conditions;

[0122] After obtaining the convergence of the image to be predicted, it is determined whether the convergence meets the set conditions. If it does, step 404 is executed; otherwise, step 405 is executed. For example, it can be determined whether the difference between the convergence and 100% is within ±5%. If so, it means that the feature difference between the image to be predicted and the corresponding target image is less than 5%.

[0123] 404. The image to be predicted is identified as the target image after reconstruction;

[0124] If the convergence meets the set conditions, the feature difference between the image to be predicted and the corresponding target image is very small. It can be considered that the image to be predicted has achieved a good image reconstruction effect. Therefore, the image to be predicted is determined as the target image after reconstruction. Then the target image can be output to help with lesion diagnosis.

[0125] 405. Continue iterative reconstruction of the image to be predicted.

[0126] If the convergence does not meet the set conditions, the feature difference between the image to be predicted and the corresponding target image is large. This indicates that the image reconstruction effect of the image to be predicted is poor, and iterative reconstruction processing of the image to be predicted needs to be continued to improve its convergence. After iterative reconstruction processing of the image to be predicted, the process returns to step 402, that is, the convergence prediction model is used again to obtain the convergence of the image to be predicted, and it is continued to determine whether the convergence meets the set conditions, until an image to be predicted with convergence meeting the set conditions is obtained, which is then output as the target image.

[0127] With this setup, the convergence prediction model can be used to detect the quantized convergence value of the image in real time during the image reconstruction process, continuously evaluate whether the current convergence of the image meets the conditions, and finally easily obtain the reconstructed image with the convergence meeting the conditions.

[0128] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0129] The above mainly describes a convergence prediction method for image reconstruction and an image reconstruction method. The following will describe a convergence prediction device for image reconstruction.

[0130] Please see Figure 5 This application provides an image reconstruction convergence prediction device, comprising:

[0131] Image acquisition module 501 is used to acquire the image to be predicted generated by iterative reconstruction.

[0132] The convergence prediction module 502 is used to input the first feature data of the image to be predicted into the trained convergence prediction model, and output the convergence of the image to be predicted through the convergence prediction model; wherein, the convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image, and the convergence prediction model is a deep learning model used to predict the convergence of image reconstruction.

[0133] In one implementation of this application, the convergence prediction device further includes:

[0134] The first image information acquisition module is used to acquire image information of the image to be predicted.

[0135] The first encoding module is used to encode the image information of the image to be predicted to obtain the first feature data.

[0136] In another implementation of this application embodiment, the convergence prediction device further includes:

[0137] The second image information acquisition module is used to acquire image information of the image to be predicted.

[0138] The second encoding module is used to encode the image information of the image to be predicted to obtain the first feature vector;

[0139] The image feature extraction module is used to extract image features from the image to be predicted, and obtain a second feature vector.

[0140] The feature fusion module is used to fuse the first feature vector and the second feature vector to obtain the first feature data.

[0141] In one implementation of this application, the convergence prediction device further includes:

[0142] The training sample acquisition module is used to acquire one-to-one corresponding sample images and reference images. The sample images and reference images are generated using the same iterative reconstruction algorithm, with the reference image having a higher number of iterations than the sample image, and the reference image corresponding to the highest convergence.

[0143] The training set data construction module is used to construct the training set data and the loss function based on the sample images and the benchmark images.

[0144] The model training module is used to train a convergence prediction model based on the training set data and by minimizing the loss function.

[0145] In one implementation of this application, the training set data construction module includes:

[0146] The first feature data generation unit is used to generate the second feature data of the sample image based on the image information and image features of the sample image;

[0147] The second feature data generation unit is used to generate the third feature data of the reference image based on the image information and image features of the reference image.

[0148] The training set data construction unit is used to construct the training set data by taking the second feature data as input and the difference between the second feature data and the third feature data as the output target.

[0149] In one implementation of this application, the first feature data generation unit includes:

[0150] The first region segmentation subunit is used to perform region segmentation processing on the sample image to obtain the first region of interest of the sample image.

[0151] The first feature extraction subunit is used to extract image features of the first region of interest and its adjacent regions to obtain the third feature vector;

[0152] The first encoding subunit is used to encode the image information of the sample image to obtain the fourth feature vector;

[0153] The first feature fusion subunit is used to fuse the third feature vector with the fourth feature vector to obtain the second feature data;

[0154] The second feature data generation unit includes:

[0155] The second region segmentation subunit is used to perform region segmentation processing on the reference image to obtain the second region of interest of the reference image;

[0156] The second feature extraction subunit is used to extract image features of the second region of interest and its adjacent regions to obtain the fifth feature vector.

[0157] The second encoding subunit is used to encode the image information of the reference image to obtain the sixth feature vector;

[0158] The second feature fusion subunit is used to fuse the fifth feature vector with the sixth feature vector to obtain the third feature data.

[0159] In one implementation of this application, the convergence prediction device further includes:

[0160] The test image acquisition module is used to acquire multiple test images from the sample images;

[0161] The test image prediction module is used to input multiple test images into the convergence prediction model and output the convergence prediction results of multiple test images through the convergence prediction model.

[0162] The confidence interval calculation module is used to calculate the confidence interval of the prediction result output by the convergence prediction model based on the fourth feature data of multiple test images, the fifth feature data of the corresponding reference image, and the convergence prediction results of multiple test images.

[0163] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the convergence prediction method for image reconstruction as shown in any of the above embodiments, or implements the image reconstruction method as shown in any of the above embodiments.

[0164] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to execute the convergence prediction method for image reconstruction as shown in any of the above embodiments, or to execute the image reconstruction method as shown in any of the above embodiments.

[0165] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the embodiments of the convergence prediction methods for image reconstruction described above, for example... Figure 1 Steps 101 to 102 shown, or steps in embodiments of the above image reconstruction method, for example... Figure 4 Steps 401 to 405 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 502 are shown.

[0166] The computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the electronic device 6.

[0167] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0168] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or memory. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0170] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0171] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0172] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0173] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0174] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0175] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0176] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0177] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A convergence prediction method for image reconstruction, characterized in that, include: Obtain the image to be predicted generated through iterative reconstruction; The first feature data of the image to be predicted is input into a trained convergence prediction model, and the convergence of the image to be predicted is output by the convergence prediction model; wherein, the convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image, and the convergence prediction model is a deep learning model used to predict the convergence of image reconstruction.

2. The method as described in claim 1, characterized in that, The first feature data is obtained in the following way: Obtain the image information of the image to be predicted; The image information of the image to be predicted is encoded to obtain the first feature data.

3. The method as described in claim 1, characterized in that, The first feature data is obtained in the following way: Obtain the image information of the image to be predicted; The image information of the image to be predicted is encoded to obtain a first feature vector; Extract the image features of the image to be predicted to obtain a second feature vector; The first feature vector and the second feature vector are fused to obtain the first feature data.

4. The method as described in claim 1, characterized in that, The training process of the convergence prediction model includes: Obtain a one-to-one corresponding sample image and a reference image; wherein the sample image and the reference image are generated using the same iterative reconstruction algorithm, the reference image has a higher number of iterations than the sample image, and the reference image corresponds to the highest convergence. Based on the sample images and the benchmark images, a training set data and a loss function are constructed. The convergence prediction model is trained based on the training set data and by minimizing the loss function.

5. The method as described in claim 4, characterized in that, The step of constructing training set data based on the sample images and the reference images includes: Based on the image information and image features of the sample image, generate the second feature data of the sample image; Based on the image information and image features of the reference image, generate the third feature data of the reference image; The training set data is constructed by using the second feature data as input and the difference between the second feature data and the third feature data as the output target.

6. The method as described in claim 5, characterized in that, The step of generating second feature data for the sample image based on the image information and image features of the sample image includes: Perform region segmentation processing on the sample image to obtain the first region of interest of the sample image; Extract the image features of the first region of interest and its adjacent regions to obtain the third feature vector; The image information of the sample image is encoded to obtain a fourth feature vector; The third feature vector is fused with the fourth feature vector to obtain the second feature data; The step of generating third feature data of the reference image based on the image information and image features of the reference image includes: Perform region segmentation processing on the reference image to obtain the second region of interest of the reference image; The image features of the second region of interest and its adjacent regions are extracted to obtain the fifth feature vector; The image information of the reference image is encoded to obtain the sixth feature vector; The fifth feature vector is fused with the sixth feature vector to obtain the third feature data.

7. The method according to any one of claims 4 to 6, characterized in that, After training and obtaining the convergence prediction model, the following steps are also included: Multiple test images are obtained from the sample images; The multiple test images are input into the convergence prediction model, and the convergence prediction model outputs the convergence prediction results of the multiple test images. Based on the fourth feature data of the multiple test images, the fifth feature data of the reference image corresponding to the multiple test images, and the convergence prediction results of the multiple test images, the confidence interval of the prediction results output by the convergence prediction model is calculated.

8. An image reconstruction method, characterized in that, include: Obtain the image to be predicted generated through iterative reconstruction; The feature data of the image to be predicted is input into a trained convergence prediction model, and the convergence of the image to be predicted is output by the convergence prediction model; wherein, the convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image, and the convergence prediction model is a deep learning model used to predict the convergence of image reconstruction. If the convergence meets the set conditions, the image to be predicted is determined as the target image after reconstruction. If the convergence does not meet the set conditions, the image to be predicted will continue to be reconstructed iteratively, and then the process will return to the step of inputting the feature data of the image to be predicted into the trained convergence prediction model.

9. A convergence prediction device for image reconstruction, characterized in that, include: The image acquisition module is used to acquire the image to be predicted generated through iterative reconstruction. The convergence prediction module is used to input the first feature data of the image to be predicted into the trained convergence prediction model, and output the convergence of the image to be predicted through the convergence prediction model; wherein, the convergence is used to characterize the feature difference between the image to be predicted and the corresponding target image, and the convergence prediction model is a deep learning model used to predict the convergence of image reconstruction.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the convergence prediction method for image reconstruction as described in any one of claims 1 to 7, or the image reconstruction method as described in claim 8.

Images