Image reconstruction model construction method and device, image generation method and device and computer equipment

Abstract

The invention relates to an image reconstruction model construction method and device, an image generation method and device and computer equipment. The method comprises the following steps: acquiring two training X-ray images, a noise-added CT image and a to-be-trained image reconstruction model; taking the current noise adding step identifier as guide data, taking the two training X-ray images and the noise adding CT image as condition data, and inputting the condition data into the to-be-trained image reconstruction model to obtain model prediction noise; calculating a model noise loss value corresponding to the difference between the representation model prediction noise and the image real noise of the noise-added CT image; the model noise loss value comprises skeleton label information for the skeleton region; and training the to-be-trained image reconstruction model according to the model noise loss value to obtain a trained image reconstruction model. By adopting the method, the image of the target object can be more effectively reconstructed, so that the three-dimensional skeleton space information of the target object can be obtained on the basis of low-radiation damage, and the definition and medical diagnosis value of the reconstructed image are further remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method, device, and computer equipment for constructing an image reconstruction model and generating images. Background Art

[0002] With the development of 3D bone model technology, the clinical application of 3D bone model is becoming more and more important. For example, bone model can be used for 3D display of scoliosis and provide 3D information during orthopedic surgery. Although traditional technology can obtain 3D bone model based on computed tomography (CT) scanning, this method will cause high radiation damage to the target object. Under low radiation damage, X-ray images can be used to view the bone state, but two-dimensional X-rays are difficult to provide three-dimensional bone space information. Therefore, traditional methods cannot obtain three-dimensional bone space information of the target object on the basis of low radiation damage. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, and computer equipment for constructing and generating an image reconstruction model that can obtain three-dimensional bone spatial information of a target object based on low radiation damage in order to address the above technical problems.

[0004] In a first aspect, the present application provides a method for constructing an image reconstruction model, comprising:

[0005] Obtain two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0006] Using the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, input into the image reconstruction model to be trained, and obtain model prediction noise;

[0007] Calculating a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; the model noise loss value includes bone label information for the bone area;

[0008] The image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

[0009] In a second aspect, the present application also provides an image generation method, comprising:

[0010] Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is trained by any image reconstruction model construction method;

[0011] Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise;

[0012] Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data;

[0013] Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

[0014] In a third aspect, the present application also provides a device for constructing an image reconstruction model, comprising:

[0015] A conditional data acquisition module, used to acquire two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0016] A predicted noise obtaining module is used to use the current noise adding step identifier as guide data, the two training X-ray images and the noisy CT image as conditional data, input them into the image reconstruction model to be trained, and obtain the model predicted noise;

[0017] A model loss calculation module, used to calculate a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image;

[0018] The reconstruction model training module is used to train the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model.

[0019] In a fourth aspect, the present application further provides an image generation device, comprising:

[0020] Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is trained by any image reconstruction model construction method;

[0021] Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise;

[0022] Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data;

[0023] Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

[0024] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Obtain two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0026] Using the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, input into the image reconstruction model to be trained, and obtain model prediction noise;

[0027] Calculating a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; the model noise loss value includes bone label information for the bone area;

[0028] The image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

[0029] as well as,

[0030] Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is obtained by training using any image reconstruction model construction method;

[0031] Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise;

[0032] Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data;

[0033] Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

[0034] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Obtain two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0036] Using the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, input into the image reconstruction model to be trained, and obtain model prediction noise;

[0037] Calculating a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; the model noise loss value includes bone label information for the bone area;

[0038] The image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

[0039] as well as,

[0040] Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is obtained by training using any image reconstruction model construction method;

[0041] Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise;

[0042] Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data;

[0043] Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

[0044] In a seventh aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0045] Obtain two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0046] Using the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, input into the image reconstruction model to be trained, and obtain model prediction noise;

[0047] Calculating a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; the model noise loss value includes bone label information for the bone area;

[0048] The image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

[0049] as well as,

[0050] Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is obtained by training using any image reconstruction model construction method;

[0051] Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise;

[0052] Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data;

[0053] Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

[0054] The above-mentioned method, device, computer equipment, storage medium and computer program product for constructing an image reconstruction model are obtained by obtaining two training X-ray images, a noisy CT image and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier; using the current noisy step identifier as guiding data, the two training X-ray images and the noisy CT image are used as conditional data and input into the image reconstruction model to be trained to obtain model prediction noise; calculating a model noise loss value corresponding to the difference between the image prediction noise and the real noise of the noisy CT image; the model noise loss value includes bone label information for the bone area; training the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model.

[0055] By introducing two X-ray images and a noisy CT image as conditional data, combined with the current noisy step identifier as the guide data, the image reconstruction model to be trained can capture image details and noise patterns based on multi-source information, thereby enhancing the accuracy of noise prediction. In particular, by introducing the bone label information of the bone area, the image reconstruction model to be trained can focus on optimizing the reconstruction effect of the bone area in complex structured data, ensuring the accurate presentation of bone edges and details. In addition, by calculating the difference between the noise predicted by the model and the actual noise, and feeding the loss value back to the model training, the reconstruction ability of the image reconstruction model to be trained is gradually optimized, and finally a trained image reconstruction model that can effectively reduce noise and retain high-quality details is formed. The model trained by the above steps can more effectively reconstruct the image of the target object to obtain the three-dimensional bone space information of the target object on the basis of low radiation damage, further significantly improving the clarity and medical diagnostic value of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] FIG1 is a diagram showing an application environment for constructing an image reconstruction model and an image generation method in one embodiment;

[0058] FIG2 is a schematic diagram of a flow chart of a method for constructing an image reconstruction model in one embodiment;

[0059] FIG3 is a schematic flow chart of a method for calculating a model noise loss value in one embodiment;

[0060] FIG4 is a schematic flow chart of a method for obtaining a bone region loss value in one embodiment;

[0061] FIG5 is a schematic diagram of a flow chart of a method for training a trained image reconstruction model in one embodiment;

[0062] FIG6 is a schematic flow chart of a method for obtaining model prediction noise in one embodiment;

[0063] FIG7 is a schematic diagram of a flow chart of an image generation method in one embodiment;

[0064] FIG8 is a schematic diagram of a process of constructing an image reconstruction model in one embodiment;

[0065] FIG9 is a schematic diagram of an image effect of a CT reconstruction process in one embodiment;

[0066] FIG10 is a structural block diagram of a device for constructing an image reconstruction model in one embodiment;

[0067] FIG11 is a structural block diagram of an image generating device in one embodiment;

[0068] FIG. 12 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] A method for constructing an image reconstruction model provided in an embodiment of the present application can be applied in an application environment as shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained from the terminal 102; the noisy CT image includes the current noisy step identifier; using the current noisy step identifier as the guide data, the two training X-ray images and the noisy CT image are used as conditional data, and input into the image reconstruction model to be trained to obtain the model prediction noise; calculate the model noise loss value corresponding to the difference between the image prediction noise and the real noise of the noisy CT image; the model noise loss value includes the bone label information for the bone area; the image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0071] In an exemplary embodiment, as shown in FIG2 , a method for constructing an image reconstruction model is provided, which is described by taking the method applied to the server in FIG1 as an example, and includes the following steps 202 to 208 . Among them:

[0072] Step 202: Obtain two training X-ray images, a noisy CT image, and a model for reconstructing the image to be trained.

[0073] Among them, training X-ray images can be additional image data used to assist in the training of CT image reconstruction models, usually obtained through X-ray imaging technology. They provide anatomical information from different perspectives or modalities, such as the structure or contour of bones, which can provide supplementary information for model training and help the model better understand and predict noise and details in CT images under different conditions.

[0074] In order to better guide the learning of the model at different training stages, a noise step identifier is added to the noisy CT image to mark the training step of the noisy CT image, that is, the number of times the noise has been added. The current noise step identifier can be a CT image obtained by artificially adding noise to the noisy CT image marked as any step. Each stage of noise or training has its unique step identifier, which is used to mark the location of the current noise or training.

[0075] The image reconstruction model to be trained can be a deep learning model that has not been fully trained. It usually learns how to effectively remove image noise and reconstruct high-quality CT images through multiple input sources (such as X-ray images and noisy CT images). It is in an initial state, waiting to gradually improve its image processing capabilities through multiple training rounds.

[0076] Specifically, two training X-ray images are obtained from the X-ray image data set corresponding to any target object. These images can provide bone structure information at different angles or different modalities as additional input information. For example, these two training X-ray images can be taken at orthogonal angles. At the same time, a CT image that has been noise-added is selected. Noise is artificially added to the image. This noisy CT image contains not only the anatomical information of the image itself, but also the noise pattern that needs to be removed. In addition, an untrained deep learning model is obtained as the image reconstruction model to be trained, ready to accept input data for training and optimization.

[0077] Step 204, using the current noise adding step identifier as the guide data, the two training X-ray images and the noisy CT image as conditional data, are input into the image reconstruction model to be trained to obtain the model prediction noise.

[0078] The model prediction noise can be the noise addition result generated by the image reconstruction model to be trained based on the input data (such as X-ray images and noisy CT images). It represents the model's estimation of the noise part in the noisy CT images, which is used for subsequent comparison and optimization with the real noise of the images, helping the model to better identify and remove noise during the training process.

[0079] Specifically, the current noise step identifier is used as the guiding data for training the image reconstruction model to be trained, and the two training X-ray images, the noisy CT image and the current noise step identifier (such as the t-th step noise identifier) ​​are input as conditions to the image reconstruction model to be trained, that is, the fused data (X2CTData) obtained by fusing the noisy CT image and the two training X-ray images are spliced ​​and passed into the three-dimensional U-net as input data. The image reconstruction model to be trained learns to understand the correlation between different image modalities from these input data, wherein the fused data is obtained by copying and transposing two orthogonal X-ray images of the bone region of the target object, respectively, to obtain the fusion of two three-dimensional data after preliminary processing. The two training X-ray images provide bone contour information other than the noisy CT image, which helps the model to better separate noise and useful image features. A noise processing mechanism is introduced into the image reconstruction model to be trained to achieve noise addition, which can be achieved by adding noise in the conditional data or integrating a noise layer in the network structure. Further, based on the conditional data of the reconstruction model after adding noise, a noise processing mechanism is introduced into the image reconstruction model to be trained to achieve noise elimination, which can be achieved by eliminating noise in the reconstruction model conditional data or integrating a noise layer in the network structure, and the noise is predicted by the output model of the image reconstruction model to be trained.

[0080] In practice, various noise generation and noise reduction techniques can be used, such as introducing random noise, using autoencoders, etc. By comparing these two outputs, we can evaluate the effect of the model on noise processing and the performance of the model under different noise levels.

[0081] Step 206 , calculating a model noise loss value corresponding to the difference between the noise predicted by the model and the real noise of the noisy CT image.

[0082] The real image noise can be the known noise value in the noisy CT image, which is usually clear when the noise is artificially added. This real noise is used to evaluate the accuracy of the model prediction noise, and the model optimization is guided by comparing the difference between the model prediction noise and the real image noise.

[0083] The model noise loss value can be the loss value calculated by comparing the model prediction noise with the real image noise. It reflects the current performance of the model in the noise removal task. The smaller the loss value, the more accurate the model prediction. This loss value not only measures the performance of the model globally, but also performs local optimization for key areas (such as bones).

[0084] Specifically, in order to accurately reconstruct, the calculation of the model noise loss value needs to go through the joint action of three types of noise. The first type is based on the logic of the diffusion model. The noise is considered to conform to the standard Gaussian distribution. The noise regression loss function uses the L1 loss. The L1 loss of noise regression between the model prediction noise and the real noise of the image is based on the model prediction noise and the real noise of the image, which fully considers the influence of noise; the second type is the projection loss between the model prediction noise and the real noise of the image. By projecting along the x, y, and z axes and calculating the L1 loss, the spatial consistency of the image is ensured; the third type is the noisy-loss specifically for the bone area. By multiplying the model prediction noise and the real noise of the image with the bone label and calculating the L1 loss, the optimization of the bone area is highlighted. By increasing the weight of the bone part, the model can more accurately reconstruct the bone structure in the CT image. The entire training process follows the conventional Diffusion Model steps, and a total of 1000 diffusion steps are performed. By comparing the model prediction noise with the known real noise of the image in the noisy CT image, the difference between the two is calculated as the model noise loss value. It is used to evaluate the accuracy of the model in the current prediction noise. The model noise loss value not only simply measures the overall noise removal effect, but also specifically refines the bone area, using the bone label information to accurately locate and optimize the noise removal in this area. Noise removal in the bone area is the most critical part of CT images, so the model needs to pay special attention to the reconstruction effect of this area. This bone label information helps the model improve its ability to handle the details of the bone structure in the loss calculation.

[0085] Step 208 , training the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model.

[0086] The trained image reconstruction model can be a mature model obtained after multiple rounds of optimization. It can more accurately remove noise from CT images and effectively reconstruct details, especially in the processing of key areas such as bones. This model has strong generalization ability and is suitable for CT image reconstruction tasks in practical applications.

[0087] Specifically, the model noise loss value is fed back to the image reconstruction model to be trained through the back propagation algorithm to update the parameters of the model. The larger the model noise loss value, the greater the error of the image reconstruction model to be trained in the current prediction, and the image reconstruction model to be trained will reduce future errors by adjusting parameters; the smaller the model noise loss value, the higher the prediction accuracy of the image reconstruction model to be trained. With continuous iterative training, the image reconstruction model to be trained will gradually learn how to better remove noise from the image, especially in the bone area. The image reconstruction model to be trained will further optimize the noise suppression in this area based on the bone label information. Through multiple rounds of training, the denoising performance and reconstruction accuracy of the image reconstruction model to be trained will be significantly improved, and finally a fully trained image reconstruction model will be obtained, which can effectively denoise noisy CT images, and excel in the restoration of bone structure and the reproduction of details, which is suitable for the image reconstruction needs of clinical diagnosis.

[0088] In the above-mentioned method for constructing an image reconstruction model, two training X-ray images, a noisy CT image and a reconstruction model of the image to be trained are obtained; the noisy CT image includes a current noisy step identifier; the current noisy step identifier is used as a guide data, and the two training X-ray images and the noisy CT image are used as conditional data and input into the image reconstruction model to be trained to obtain model prediction noise; the model noise loss value corresponding to the difference between the image prediction noise and the real noise of the noisy CT image is calculated; the model noise loss value includes bone label information for the bone area; the image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

[0089] By introducing two X-ray images and a noisy CT image as conditional data, combined with the current noisy step identifier as the guide data, the image reconstruction model to be trained can capture image details and noise patterns based on multi-source information, thereby enhancing the accuracy of noise prediction. In particular, by introducing the bone label information of the bone area, the image reconstruction model to be trained can focus on optimizing the reconstruction effect of the bone area in complex structured data, ensuring the accurate presentation of bone edges and details. In addition, by calculating the difference between the noise predicted by the model and the actual noise, and feeding the loss value back to the model training, the reconstruction ability of the image reconstruction model to be trained is gradually optimized, and finally a trained image reconstruction model that can effectively reduce noise and retain high-quality details is formed. The model trained by the above steps can more effectively reconstruct the image of the target object to obtain the three-dimensional bone space information of the target object on the basis of low radiation damage, further significantly improving the clarity and medical diagnostic value of the reconstructed image.

[0090] In an exemplary embodiment, as shown in FIG3 , calculating the model noise loss value corresponding to the difference between the noise predicted by the model and the real noise of the noisy CT image includes steps 302 to 306. In which:

[0091] Step 302: Perform noise regression on the model prediction noise and the real image noise to obtain a noise reconstruction loss value.

[0092] The noise reconstruction loss value may be a loss value that fully considers the influence of noise.

[0093] Specifically, the image reconstruction model to be trained will perform noise regression based on the modeling logic of the Diffusion Model, and assume that the model prediction noise and the real image noise conform to the standard Gaussian distribution. In order to effectively remove noise from the image reconstruction model to be trained, the L1 loss function is used in the process. Its core purpose is to minimize the noise difference between the model prediction noise generated by the model and the real image noise of the noisy CT image. Finally, the image reconstruction model to be trained outputs the noise reconstruction loss value (Reconstruction Loss). Among them, the L1 loss function is mainly based on calculating the absolute difference between the predicted value and the real value, ensuring that the model gradually converges and improves the quality of the reconstructed image.

[0094] Step 304 , calculating the difference between the projections of the model predicted noise and the real image noise on the coordinate axis to obtain a noise projection loss value.

[0095] The projection loss value may be the projection loss of three dimensions in a three-dimensional coordinate system, which is used to constrain the consistency of the space.

[0096] Specifically, the two three-dimensional volume data of model prediction noise and image real noise are projected according to each coordinate axis in the three-dimensional coordinate system. This process can be achieved by performing maximum intensity projection (MIP) or other projection methods on each coordinate axis. Three sets of projection model prediction noise and three sets of projection image real noise are obtained, corresponding to the X, Y and Z axes respectively. During the projection process, appropriate projection methods and parameters can be selected to best retain important information in the image. This involves the selection of different projection algorithms and angles, depending on specific application requirements. After obtaining the prediction noise of each projection model and the real noise of each projection image, the performance of these projection images on different coordinate axes can be further analyzed.

[0097] Select a specific coordinate axis (for example, the X-axis, Y-axis, or Z-axis), determine the projection model prediction noise and the real noise of the projection image on the corresponding coordinate axis, and calculate the sum of the projection model prediction noise and the real noise of the projection image to calculate the L1 loss as the projection loss value. This calculation can also use other loss functions, such as mean square error (MSE) or other suitable indicators to quantify the image reconstruction performance of the model on a specific coordinate axis. In actual operation, different loss functions can be used, depending on the requirements of the task and the required evaluation indicators. This process can be performed by traversing different coordinate axes and applying the corresponding loss function to obtain the projection loss value in each direction, and further fusing the projection loss values ​​in each direction to obtain the noise projection loss value.

[0098] Step 306, calculating the difference between the model predicted noise and the real image noise in the bone region to obtain the bone region loss value.

[0099] Among them, the bone area loss value can be a loss value calculated only for the bone-related part, which is equivalent to adding a weight to the bone area.

[0100] Specifically, the bone label information is used to increase the characteristics of the model prediction noise and the real image noise in the bone area, and the difference between the model prediction noise and the real image noise in the bone area after the added characteristics is further calculated. The L1 loss function is usually used, that is, the absolute value error between them is calculated to obtain the bone area loss value. This loss value reflects the effect of the model in removing noise in the bone area. The smaller the error, the more accurate the reconstruction of the model in this area.

[0101] In this embodiment, by performing noise regression on the model predicted noise and the real noise, the overall noise removal effect of the model can be accurately evaluated to ensure the quality of the reconstructed image. Further, by calculating the noise projection loss value based on the projection difference on the coordinate axis, the consistency of the image in the spatial structure can be guaranteed, and the problem of uneven local noise processing in the three-dimensional image can be avoided. Regarding the loss calculation of the bone area, the model specially optimizes the noise removal of the bone area to ensure higher clarity and accuracy in the key bone area, thereby improving the reliability of CT images in clinical diagnosis. Combining the above three different loss values, the multi-dimensional noise loss calculation is realized, which greatly improves the accuracy of CT image reconstruction.

[0102] In an exemplary embodiment, the difference between the noise predicted by the calculation model and the real noise of the image in the bone region is obtained to obtain the bone region loss value, including:

[0103] According to the bone label information, the model prediction noise and the real image noise are increased in the prominence of the bone area in the training X-ray image to obtain the bone area loss value.

[0104] The skeleton label information may be a segmentation mask with a label after the skeleton region is annotated.

[0105] Specifically, by using the bone label information pre-annotated in the bone area and fusing it with the model prediction noise, the prominence of the model prediction noise in the bone area of ​​the training X-ray image is improved, and the noise information related to the bone is screened out. Similarly, by using the bone label information pre-annotated in the bone area and fusing it with the real image noise, the prominence of the real image noise in the bone area of ​​the training X-ray image is improved. Then, the L1 loss between these two prominences, namely "noisy-loss", is calculated to obtain the bone area loss value to quantify the noise removal accuracy of the model in the bone area.

[0106] In this embodiment, by annotating the training X-ray images, clarifying the location of the bone area, and increasing the prominence of the bone area during the noise removal process, the model can pay special attention to this key area. When calculating the difference between the model-predicted noise and the actual noise, the bone area loss value is enhanced, thereby effectively improving the ability to reconstruct the details of the bone area. Therefore, the model can more accurately remove the noise in the bone area, significantly improve the reconstruction accuracy of CT images in the bone area, and meet the high clinical requirements for bone details.

[0107] In an exemplary embodiment, as shown in FIG4 , according to the bone label information, the prominence of the model prediction noise and the real image noise in the bone region in the training X-ray image is increased to obtain the bone region loss value, including steps 402 to 406. Wherein:

[0108] Step 402, multiply the model prediction noise and the skeleton label information to obtain the first skeleton label noise.

[0109] Specifically, by introducing the skeleton label information that is pre-processed or annotated and accurately identifies the pixel position of the skeleton area in the skeleton area, the model prediction noise is multiplied pixel by pixel with the skeleton label information, so that the image reconstruction model to be trained can focus on the model prediction noise in the skeleton area. The result of the multiplication operation is that the model prediction noise value in the non-skeleton area is set to zero, while the model prediction noise in the skeleton area is retained and amplified, so that the prediction result of the image reconstruction model to be trained is more prominent in the model prediction noise in the skeleton area. This method effectively separates the part of the model prediction noise in the skeleton area, forming the so-called first skeleton label noise, ensuring that the model can be targeted to optimize the skeleton area during the training process.

[0110] Step 404: multiply the real image noise by the skeleton label information to obtain a second skeleton label noise.

[0111] Specifically, by introducing the skeleton label information of the pixel position of the skeleton region that is generated by pre-processing or annotation in the skeleton region, the real image noise and the skeleton label information are multiplied pixel by pixel, so that the image reconstruction model to be trained can focus on the real image noise in the skeleton region. The result of the multiplication operation is that the real image noise value of the non-skeleton region is reset to zero, and the real image noise of the skeleton region is retained and amplified, so that the prediction result of the image reconstruction model to be trained is more prominent in the real image noise of the skeleton region. This mode effectively separates the real image noise of that part of the model in the skeleton region, forming the so-called second skeleton label noise.

[0112] Step 406, calculating the difference between the first skeleton label noise and the second skeleton label noise to obtain the skeleton region loss value.

[0113] Specifically, the L1 loss is calculated for the first skeleton label noise and the second skeleton label noise. The L1 loss function quantifies the accuracy of the model in removing noise in the skeleton area by calculating the absolute difference between the first skeleton label noise and the second skeleton label noise. This loss is called "noisy-loss", which specifically evaluates the noise removal effect in the skeleton area. In order to further improve the generation effect of the model in the skeleton area, a weight can be added to this loss function so that the model gives more optimization attention to the skeleton area during the training process, thereby improving the reconstruction quality of the skeleton area and obtaining the skeleton area loss value.

[0114] In the present embodiment, by multiplying the model prediction noise and the real noise with the skeleton label respectively, the noise information of the skeleton area is accurately extracted, and the interference caused by the global noise processing is avoided. The difference between the predicted noise and the real noise in the skeleton area is calculated to obtain the skeleton area loss value. This process ensures that the model focuses on optimizing the noise removal of the skeleton area during training, making the image details of the skeleton area clearer, and effectively enhancing the noise removal ability of the model in the skeleton area, thereby significantly improving the accuracy and clinical practicality of reconstructing CT images on the skeleton details.

[0115] In an exemplary embodiment, as shown in FIG5 , the image reconstruction model to be trained is trained according to the model noise loss value to obtain the trained image reconstruction model, including steps 502 to 506. In which:

[0116] Step 502 : adjusting the model parameters of the image reconstruction model to be trained according to the model noise loss value until the model noise loss value is less than a preset condition, thereby obtaining an adjusted image reconstruction model.

[0117] The adjusted image reconstruction model may be a new model obtained by adjusting the model parameters of the image reconstruction model to be trained according to the model noise loss value under the current step identifier.

[0118] Specifically, according to the model noise loss value, using the back propagation algorithm, the image reconstruction model to be trained will adjust according to the contribution of each parameter to the model noise loss value, thereby gradually reducing the model noise loss value. In each iteration of this process, the image reconstruction model to be trained will continuously predict the noise and calculate the new model noise loss value, and continue to optimize the model parameters according to the new model noise loss value. Training will continue until the model noise loss value drops below the preset condition, which means that the image reconstruction model to be trained can accurately predict and remove noise. At this stage, the image reconstruction model to be trained gradually strengthens its denoising ability in processing noisy CT images, especially in the bone area of ​​focus. After multiple adjustments, the image reconstruction model to be trained can effectively restore the details and structures in the CT image under the current step identification, and obtain an adjusted image reconstruction model.

[0119] Step 504: Use the adjusted image reconstruction model as the image reconstruction model to be trained, and obtain the noisy CT image marked in the previous noisy step as the noisy CT image marked in the current noisy step.

[0120] Among them, the noisy CT image marked by the previous noisy step can be marked as the noisy operation of the previous step relative to the current noisy step mark. If the tth step is used as the current noisy step mark, the t-1th step is the previous noisy step mark, and then the t-1th step corresponds to the CT image processed by artificial noisy processing. Each stage of noisy or training has its unique step mark, which is used to mark the location of the current noisy or training.

[0121] Specifically, the adjusted image reconstruction model will be used as the basis for the next round of training and become the new image reconstruction model to be trained. It is worth noting that because the adjusted image reconstruction model after one optimization can learn and adjust faster and more accurately in the new training round, the image reconstruction model to be trained will not start from scratch in this process, but continue to learn based on the adjusted parameters to improve efficiency and enhance denoising capabilities. At the same time, the data part is also updated, replacing the noisy CT image with the previous noise step identifier (step t-1) with the noisy CT image with the current noise step identifier (step t). In this way, the model uses different noisy image data in different training stages to ensure that the model can gradually cope with more complex or different types of noise scenes.

[0122] Step 506, returns to execute the step of using the current denoising step identifier as the guide data, inputting the two training X-ray images and the noisy CT image as conditional data into the image reconstruction model to be trained, and obtaining the model prediction noise step, until the current denoising step identifier triggers the preset value, and obtaining the trained image reconstruction model.

[0123] Among them, the current noise addition step identifier triggers the preset value can be the step of adding noise to the CT image, and it is also the step of reducing noise in the CT image, that is, the diffusion step of model training, which is generally set when the model training starts.

[0124] Specifically, two training X-ray images, a noisy CT image, and a noisy CT image with a current noisy step identifier replaced by a noisy CT image with a previous noisy step identifier are used as conditional data and re-entered into the image reconstruction model to be trained that is replaced by the adjusted image reconstruction model. Through these conditional data, the model will generate new model prediction noise again and conduct the next round of training and optimization. This process is repeated continuously, and the denoising performance of the model is gradually optimized through multiple iterations. In each round of training, the model is learning how to predict the noise of the image more accurately, and the model noise loss value is continuously reduced through the aforementioned parameter adjustment process. The entire cycle continues until the current noisy step identifier triggers a preset value (for example, a certain number of training steps is reached or the loss value is lower than a certain threshold). At this time, the training process is terminated and the trained image reconstruction model is obtained. At this time, the trained image reconstruction model already has a strong denoising capability, can effectively process the noise of CT images under various conditions, and generate high-quality reconstructed images, especially in key bone areas.

[0125] In this embodiment, by continuously adjusting the model parameters until the noise loss value is lower than the preset conditions, it is ensured that the model can accurately remove noise and optimize the image reconstruction effect. In each iteration, the adjusted model is combined with the updated noisy CT image for further training to ensure that the model can adapt to different noise conditions at different training stages. By using a loop to perform multi-step training and control the noisy images that mark different steps, the model gradually improves the denoising and reconstruction capabilities, and ultimately obtains a more accurate image reconstruction model. Finally, under the condition of dynamic parameter adjustment, the accuracy of the image reconstruction model is gradually optimized so that the model can generate clearer CT images with less noise, significantly improving its clinical application value.

[0126] In an exemplary embodiment, as shown in FIG6 , the current noise addition step identifier is used as the guide data, two training X-ray images and the noisy CT image are used as conditional data, and are input into the image reconstruction model to be trained to obtain the model prediction noise, including steps 602 to 604. Among them:

[0127] Step 602: fuse the two training X-ray images to obtain fused data.

[0128] The fused data may be obtained by respectively copying and transposing two orthogonal X-ray images of the bone region of the target object, and fusing two preliminarily processed three-dimensional data.

[0129] Specifically, each training X-ray image needs to pass through a Transposed_and_Add (i.e., "transposed convolution and addition" module or "upsampling and overlay" module) to convert it into a 3D image and add it. Assuming that the set CT image is 128x128x128 (corresponding to the x-axis, y-axis, and z-axis, respectively), the first training X-ray image can be 128x128 (corresponding to the y-axis and z-axis, respectively) or other values, and the second training X-ray image can be 128x128 (corresponding to the x-axis and z-axis, respectively) or other values. Each training X-ray image can be regarded as a slice of a 3DCT image. The conversion method is to copy it n times along the shooting axis of the X-ray image, where n is the number of slices of the target CT on this axis. Taking the above data as an example, it is copied 128 times. Therefore, we get:

[0130] The first training X-ray image [size 128ⅹ128 (y axis, z axis)] → the first CT image [size 128ⅹ128ⅹ128 (y axis, z axis, x axis)],

[0131] The second training X-ray image [size 128ⅹ128 (x-axis, z-axis)] → the second CT image [size 128ⅹ128ⅹ128 (x-axis, z-axis, y-axis)].

[0132] After the two training X-ray images are transformed into three dimensions, the data of the first transformed image and the second transformed image need to be merged and reconstructed. Different views reflect the conditions of the pelvis in different directions. For example, the first training X-ray image reflects the sagittal characteristics, while the second training X-ray image reflects the coronal characteristics. Only by combining these two phases can the size information of the vertebrae be accurately captured. Therefore, through the view consistency operation, the view direction of the two training X-ray images is adjusted to be consistent with CT using the transposition operation, as shown below:

[0133] First training X-ray image CT [Size 128ⅹ128ⅹ128 (y axis, z axis, x axis)] → Permute (Xray1 CT )[Size 128ⅹ128ⅹ128 (x-axis, y-axis, z-axis)],

[0134] Second training X-ray image CT [Size 128ⅹ128ⅹ128 (x-axis, z-axis, y-axis)] → Permute (Xray2 CT )[size 128ⅹ128ⅹ128 (x-axis, y-axis, z-axis)].

[0135] After the two view data are adjusted in the same spatial coordinate direction, they are fused by direct addition operation as the fused data of the input DiffusionModel. The specific steps are shown in the following formula:

[0136] Fusion data (X2CTData) = Permute(Xray1CT)+Permute(Xray2CT).

[0137] Step 604, using the current noise adding step identifier as guide data, the fused data and the noisy CT image as conditional data, input into the image reconstruction model to be trained, and obtain the model prediction noise.

[0138] Specifically, the current noise step identifier is used as the guide data, and the fused data and the noisy CT image are used as conditional data and input into the image reconstruction model to be trained, wherein the fused data contains additional information obtained from the training X-ray image through dimensional transformation and view data fusion, which can provide the model with features of different modalities and help the model better distinguish useful information and noise in the image. By inputting these conditional data into the image reconstruction model to be trained, the image reconstruction model to be trained will predict the noise part in the current CT image based on these conditional data, that is, the image reconstruction model to be trained, under the guidance of the current noise step identifier, extracts features from the fused data and the noisy image to generate model prediction noise.

[0139] In this embodiment, the fused data combines the features of X-ray images from different angles or modalities, providing richer structural information, especially clearer details in the bone area. The fused data, the noisy CT image, and the current noisy step identifier are used as conditional inputs to further guide the model to accurately extract key information during the denoising process at different stages. By fusing different data, the input information of the image reconstruction model is effectively enhanced, which helps the subsequent model to predict noise more accurately and improve the overall quality of image reconstruction, especially in terms of detail preservation and noise removal.

[0140] The embodiment of the present application also provides an image generation method, which can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the first reconstruction data, two real X-ray images and the trained image reconstruction model from the terminal 102; the trained image reconstruction model is trained by any method for constructing an image reconstruction model; the first reconstruction data and the two real X-ray images are used as conditional data, input into the trained image reconstruction model, and obtain the first reconstruction noise; calculate the difference between the first reconstruction noise and the first reconstruction data, and obtain the second reconstruction data; use the second reconstruction data as the first reconstruction data, and return to execute the first reconstruction data and the two real X-ray images as conditional data, input into the trained image reconstruction model, and obtain the first reconstruction noise step, until the reconstruction step triggers the preset value, and outputs the reconstruction data corresponding to the triggering preset value as the reconstructed CT image. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0141] In an exemplary embodiment, as shown in FIG7 , a method for constructing an image reconstruction model is provided, and the method is applied to the server in FIG1 as an example for explanation, including the following steps 702 to 708. Among them:

[0142] Step 702, obtaining first reconstruction data, two real X-ray images and a trained image reconstruction model.

[0143] The first reconstruction data may be white noise data set for image reconstruction.

[0144] The real X-ray image may be an X-ray image captured from the object to be reconstructed.

[0145] Specifically, real X-ray images are obtained from the X-ray image data set corresponding to the object with image reconstruction requirements. These images can provide bone structure information of different angles or different modes as additional input information. For example, these two real X-ray images can be taken at orthogonal angles. At the same time, according to the image reconstruction requirements, the first reconstruction data is set, wherein the first reconstruction data is set white noise data that conforms to the reasoning logic of the Diffusion Model. Similarly, a trained image reconstruction model suitable for the image reconstruction requirements is retrieved from the storage medium of the server 104.

[0146] Step 704 , input the first reconstruction data and two real X-ray images as conditional data into the trained image reconstruction model to obtain the first reconstruction noise.

[0147] The first reconstruction noise may be the predicted noise obtained in the process of reconstructing two X-ray images in an actual application by using a trained image reconstruction model.

[0148] Specifically, the first reconstruction data and two real X-ray images are used as conditional data and input into the trained image reconstruction model together. The first reconstruction data is used as white noise data, which is usually 3D noise randomly sampled from a standard normal distribution, and the two X-ray images provide the structural information required for CT image generation. In each step, the trained image reconstruction model guides the denoising process by combining X2CTData (conditional data from the first reconstruction data and the two real X-ray images). This means that the trained image reconstruction model generates a noise prediction result, namely the first reconstruction noise, by gradually removing noise and combining the structural information in the X-ray images.

[0149] Step 706: Calculate the difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data.

[0150] The second reconstruction data may be the difference between the noise predicted by the first reconstruction and the white noise.

[0151] Specifically, the trained image reconstruction model calculates the difference between the predicted first reconstruction noise and the set first reconstruction data (white noise data), that is, the first reconstruction data (white noise data) is removed from the predicted first reconstruction noise to obtain the second reconstruction data.

[0152] Step 708, using the second reconstructed data as the first reconstructed data, returning to the step of using the first reconstructed data and two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the triggering preset value as the reconstructed CT image.

[0153] The reconstruction step triggering preset value may be a preset value set for the number of reconstruction steps of the trained image reconstruction model, which is generally the number of steps for removing noise in the reconstruction, that is, the number of diffusion steps.

[0154] The reconstructed CT image may be obtained by reconstructing two real X-ray images using a trained image reconstruction model to obtain reconstructed data similar to the real CT image.

[0155] Specifically, since the reconstruction process of the trained image reconstruction model enters an iterative loop, the generated second reconstruction data will be used as the first reconstruction data for the next round of reasoning, and will be input into the image reconstruction model again together with the two real X-ray images. In each iteration, the trained image reconstruction model generates new second reconstruction noise based on the current reconstruction data, and corrects the reconstruction result by the difference between the new second reconstruction noise and the second reconstruction data, so that it gradually approaches the real CT image. This process will be repeated until the preset value of the reconstruction step is triggered (such as a fixed number of reconstruction steps or a specific image quality is achieved). Finally, when the stop condition is met, the trained image reconstruction model outputs the final reconstruction data, that is, the generated reconstructed CT image. This means that through multiple iterative denoising, the model successfully converts the two real X-ray images into the required CT images.

[0156] In this embodiment, by using the trained image reconstruction model, the initial reconstruction data and two real X-ray images are used as conditional inputs to generate a preliminary first reconstruction noise. Then, the difference between the first reconstruction noise and the first reconstruction data is calculated to obtain more accurate second reconstruction data, which is repeatedly input into the model for multiple iterative processing. With each iteration, the reconstructed data is gradually denoised and optimized, and the quality of the CT image is gradually optimized until the preset number of steps or accuracy requirements are met. This process significantly improves the clarity and accuracy of the final generated CT image, ensuring that the image reconstruction process can effectively remove noise while retaining key structural information, which is particularly suitable for high-quality image generation in clinical applications.

[0157] In a specific embodiment, a schematic diagram of the image reconstruction model to be trained is shown in FIG8. First, the input initial 3D image data X0 is subjected to a process of gradually adding noise to generate a series of intermediate state images X t−1 , X t , until the final high-noise image X T. During the training process, the noise generation module will project the noisy image along the x, y, and z directions to generate three noisy projection images. These projection images will be compared with the real X-ray images Xray1 and Xray2 generated by projection technology, and the projection loss and L1 loss will be calculated to adjust the noise addition process and ensure that the model can gradually learn a more accurate noise removal strategy. Next, the noisy image will enter the U-net network structure, which extracts the deep features of the image through the encoder-decoder architecture and generates a denoised reconstructed image "X2noise-net". This denoised image is further compared with Xray1 and Xray2, and finally the image quality is improved through the superposition step to generate a clearer X-ray image. The image reconstruction model to be trained is repeated many times until the image reconstruction model to be trained meets the requirements, and the trained image reconstruction model can be used to complete the entire process of image reconstruction and noise removal. As shown in Figure 9, this figure is a CT reconstruction based on orthogonal X-ray images. (a) of Figure 9 can be regarded as Xray1 in model Figure 8, (b) can be regarded as Xray2 in model Figure 8, and (e, f, g) are the axial image, mid-sagittal image and mid-coronal image of the reconstructed CT image respectively. (c, d) are the rendering results of the reconstructed CT image using 3D image rendering technology, which are mainly used to show the effect in 3D. (h, i, j) are the axial image, mid-sagittal image and mid-coronal image of the CT scan image respectively. By comparing the differences between the reconstructed CT image and the CT scan image, it can be seen that the visualization effect after reconstruction using the reconstruction method is very good.

[0158] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0159] Based on the same inventive concept, the embodiment of the present application also provides a device for constructing an image reconstruction model for implementing the method for constructing an image reconstruction model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the device for constructing one or more image reconstruction models provided below can refer to the limitations of the method for constructing an image reconstruction model above, and will not be repeated here.

[0160] In an exemplary embodiment, as shown in FIG10 , a device for constructing an image reconstruction model is provided, comprising: a conditional data acquisition module 1002 , a predicted noise acquisition module 1004 , a model loss calculation module 1006 and a reconstruction model training module 1008 , wherein:

[0161] The conditional data acquisition module 1002 is used to acquire two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier;

[0162] The predicted noise obtaining module 1004 is used to use the current noise adding step identifier as the guide data, and the two training X-ray images and the noisy CT image as the conditional data, and input them into the image reconstruction model to be trained to obtain the model predicted noise;

[0163] A model loss calculation module 1006 is used to calculate a model noise loss value corresponding to the difference between the noise predicted by the model and the real noise of the noisy CT image;

[0164] The reconstruction model training module 1008 is used to train the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model.

[0165] In one embodiment, the model loss calculation module 1006 is also used to perform noise regression on the model predicted noise and the real image noise to obtain a noise reconstruction loss value; calculate the difference between the projections of the model predicted noise and the real image noise on the coordinate axis to obtain a noise projection loss value; calculate the difference between the model predicted noise and the real image noise in the bone area to obtain a bone area loss value.

[0166] In one embodiment, the model loss calculation module 1006 is also used to increase the prominence of the model prediction noise and the real image noise in the bone area in the training X-ray image according to the bone label information to obtain the bone area loss value; the bone label information is obtained by annotating the training X-ray image.

[0167] In one embodiment, the model loss calculation module 1006 is also used to multiply the model prediction noise with the bone label information to obtain the first bone label noise; multiply the real image noise with the bone label information to obtain the second bone label noise; calculate the difference between the first bone label noise and the second bone label noise to obtain the bone area loss value.

[0168] In one embodiment, the reconstruction model training module 1008 is also used to adjust the model parameters of the image reconstruction model to be trained according to the model noise loss value until the model noise loss value is less than a preset condition, thereby obtaining an adjusted image reconstruction model; using the adjusted image reconstruction model as the image reconstruction model to be trained, and using the noisy CT image identified by the previous noisy step as the noisy CT image identified by the current noisy step; returning to execute the step of using the current noisy step identifier as the guide data, inputting the two training X-ray images and the noisy CT image as conditional data into the image reconstruction model to be trained, thereby obtaining a model prediction noise step, until the current noisy step identifier triggers the preset value, thereby obtaining a trained image reconstruction model.

[0169] In one embodiment, the predicted noise obtaining module 1004 is also used to fuse two training X-ray images to obtain fused data; using the current noise adding step identifier as the guide data, the fused data and the noisy CT image are used as conditional data, and are input into the image reconstruction model to be trained to obtain the model predicted noise.

[0170] Based on the same inventive concept, the embodiment of the present application also provides an image generation device for implementing the above-mentioned image generation method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image generation device embodiments provided below can refer to the above-mentioned limitations on an image generation method, and will not be repeated here.

[0171] In an exemplary embodiment, as shown in FIG11 , an image generation device is provided, comprising: a reconstruction data acquisition module 1102 and a CT image reconstruction module 1104, wherein:

[0172] A reconstruction data acquisition module 1102, used to acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is obtained by training using any one of the image reconstruction model construction methods according to claims 1 to 6;

[0173] The CT image reconstruction module 1104 is used to input the first reconstruction data and two real X-ray images as conditional data into the trained image reconstruction model to obtain a first reconstruction noise;

[0174] The CT image reconstruction module 1104 is further used to calculate the difference between the first reconstruction noise and the first reconstruction data to obtain the second reconstruction data;

[0175] The CT image reconstruction module 1104 is also used to use the second reconstructed data as the first reconstructed data, return to execute the step of using the first reconstructed data and two real X-ray images as conditional data, input them into the trained image reconstruction model, and obtain the first reconstruction noise, until the reconstruction step triggers a preset value, and outputs the reconstruction data corresponding to the triggering preset value as the reconstructed CT image.

[0176] Each module in the above-mentioned device for constructing an image reconstruction model and an image generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0177] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG12. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store server data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for constructing an image reconstruction model and an image generation method are implemented.

[0178] Those skilled in the art will understand that the structure shown in FIG. 12 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0180] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0181] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0184] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing an image reconstruction model, characterized in that: The method comprises: Obtain two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier; Using the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, input into the image reconstruction model to be trained, and obtain model prediction noise; Calculating a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; the model noise loss value includes bone label information for the bone area; The image reconstruction model to be trained is trained according to the model noise loss value to obtain a trained image reconstruction model.

2. The method according to claim 1, characterized in that The model noise loss value includes a noise reconstruction loss value, a noise projection loss value, and a bone region loss value; the calculation of the model noise loss value corresponding to the difference between the model prediction noise and the real image noise of the noisy CT image includes: Performing noise regression on the model prediction noise and the real noise of the image to obtain the noise reconstruction loss value; Calculate the difference between the projection of the model predicted noise and the real noise of the image on the coordinate axis to obtain a noise projection loss value; The difference between the model prediction noise and the real image noise in the bone region is calculated to obtain the bone region loss value.

3. The method according to claim 2, characterized in that The calculating the difference between the model prediction noise and the real image noise in the bone region to obtain the bone region loss value includes: According to the bone label information, the prominence of the model prediction noise and the real image noise in the bone area of ​​the training X-ray image is increased to obtain the bone area loss value; the bone label information is obtained by annotating the training X-ray image.

4. The method according to claim 3, characterized in that The step of increasing the prominence of the model prediction noise and the real image noise in the bone region of the training X-ray image according to the bone label information to obtain the bone region loss value includes: Multiplying the model prediction noise by the skeleton label information to obtain a first skeleton label noise; Multiplying the real image noise by the skeleton label information to obtain a second skeleton label noise; The difference between the first skeleton label noise and the second skeleton label noise is calculated to obtain the skeleton region loss value.

5. The method according to claim 1, characterized in that The step of training the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model comprises: According to the model noise loss value, the model parameters of the image reconstruction model to be trained are adjusted until the model noise loss value is less than a preset condition, thereby obtaining an adjusted image reconstruction model; Using the adjusted image reconstruction model as the image reconstruction model to be trained, and obtaining the noisy CT image marked in the previous noisy step as the noisy CT image marked in the current noisy step; Return to executing the step of using the current denoising step identifier as the guiding data, inputting the two training X-ray images and the noisy CT image as conditional data into the image reconstruction model to be trained, and obtaining the model predicting noise, until the current denoising step identifier triggers the preset value, thereby obtaining the trained image reconstruction model.

6. The method according to claim 1, characterized in that The method uses the current noise adding step identifier as guide data, the two training X-ray images and the noise adding CT image as conditional data, and inputs them into the image reconstruction model to be trained to obtain the model prediction noise, including: fusing the two training X-ray images to obtain fused data; The current noise adding step identifier is used as guiding data, and the fused data and the noise adding CT image are used as conditional data, which are input into the image reconstruction model to be trained to obtain the model prediction noise.

7. An image generation method, characterized in that: The method comprises: Acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is obtained by training using any one of the image reconstruction model construction methods according to claims 1 to 6; Input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain first reconstruction noise; Calculating a difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data; Use the second reconstructed data as the first reconstructed data, return to the step of using the first reconstructed data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and outputting the reconstruction data corresponding to the preset value as the reconstructed CT image.

8. A device for constructing an image reconstruction model, characterized in that: The device comprises: A conditional data acquisition module, used to acquire two training X-ray images, a noisy CT image, and a reconstruction model of the image to be trained; the noisy CT image includes a current noisy step identifier; A predicted noise obtaining module is used to use the current noise adding step identifier as guide data, the two training X-ray images and the noisy CT image as conditional data, input them into the image reconstruction model to be trained, and obtain the model predicted noise; A model loss calculation module, used to calculate a model noise loss value corresponding to a difference between the model prediction noise and the real image noise of the noisy CT image; The reconstruction model training module is used to train the image reconstruction model to be trained according to the model noise loss value to obtain a trained image reconstruction model.

9. An image generating device, characterized in that: The method comprises: A reconstruction data acquisition module, used to acquire first reconstruction data, two real X-ray images and a trained image reconstruction model; the trained image reconstruction model is trained by any one of the image reconstruction model construction methods according to claims 1 to 6; a CT image reconstruction module, configured to input the first reconstruction data and the two real X-ray images as conditional data into the trained image reconstruction model to obtain a first reconstruction noise; The CT image reconstruction module is further used to calculate the difference between the first reconstruction noise and the first reconstruction data to obtain second reconstruction data; The CT image reconstruction module is also used to use the second reconstruction data as the first reconstruction data, return to execute the step of using the first reconstruction data and the two real X-ray images as conditional data, inputting them into the trained image reconstruction model, and obtaining the first reconstruction noise, until the reconstruction step triggers a preset value, and output the reconstruction data corresponding to the preset value as the reconstructed CT image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images