Disk three-dimensional reconstruction method and system based on distillation sampling and context matching

Abstract

The invention discloses a three-dimensional reconstruction method and system based on distillation sampling and context matching, and relates to the technical field of three-dimensional reconstruction and distillation learning. According to the method, firstly, the influence of equipment difference and individual difference on data quality is effectively eliminated through screening of the OCT image containing the cup and disc structure, center cutting, normalization processing and mask generation; besides, a generative model combining a noise scheduler and a UNet network is introduced, relative depth calculation is introduced by using internal and external parameters of a camera, and the model can accurately learn medical image feature distribution through a progressive noise adding and denoising mechanism. According to the method, the consistency between the two-dimensional images is measured by using a distillation sampling method, and more images are generated based on a context matching thought and are used for further calibrating the context relationship, so that the relationship between the two-dimensional images is processed more accurately. In this way, the problem of artifacts among the reconstructed three-dimensional images can be solved, and the fine degree of the three-dimensional images can be improved to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical fields of three-dimensional reconstruction and distillation learning, and particularly relates to a three-dimensional reconstruction method and system for cup and disc based on distillation sampling and context matching. Background Art

[0002] With the development of computer image technology and medical imaging equipment, the concept of digital medicine has emerged in real clinical medicine and is applied to digital diagnosis and treatment technologies, which involves computer technologies such as artificial intelligence technology, intelligent recognition algorithms, and three-dimensional reconstruction technology, and technical fields such as medical imaging, computer graphics, biomedicine, and structured light. In many medical scenarios, three-dimensional medical images (i.e., medical images after three-dimensional reconstruction) are applied in the fields of clinical diagnosis, identification of lesion tissues and organs, robot-assisted surgery, and intelligent medical systems. At present, the key task of three-dimensional medical images is the visualization of the target tissue, that is, to restore the three-dimensional structure of the target tissue, and then to visualize, locate, and evaluate the structure of the target lesion, so as to achieve modern high-intelligent medical diagnosis.

[0003] Three-dimensional medical images can show three-dimensional information that cannot be expressed in two-dimensional data. In the diagnosis of glaucoma, clinically, the vertical cup-to-disc area ratio is selected as an important indicator to measure the lesion of the cup and disc. However, this indicator cannot measure the depth lesion and the inner wall shape lesion of the cup and disc, which means that there is an inaccurate problem in measuring the incidence of glaucoma. Therefore, the diagnostic error caused by the lack of such three-dimensional information needs to be corrected by three-dimensional reconstruction. Facing the cup and disc reconstruction task, three-dimensional medical images are needed to assist in the diagnosis.

[0004] Traditional three-dimensional reconstruction algorithms are reconstructed based on statistical and filtering methods. First, relatively high-quality images are obtained through preprocessing methods such as image measurement, and then the two-dimensional images are mapped into three-dimensional space through mapping. This method filters out noise from the original data of the picture, then calculates the statistics of the noise, and synthesizes certain parameters based on this. In terms of effect, the calculation cost of this method is not high, the demand for computing resources is not large, and the quality of the generated pictures is relatively stable. However, the three-dimensional images reconstructed by this method will inevitably lose some physical and biological properties, and there are missing situations in obtaining some diagnosis and treatment information, and it is irreversible.

[0005] Another existing reconstruction method is 3D reconstruction introduced by deep learning. Due to the end-to-end characteristics and automatic learning ability of deep learning models, the obtained data can be made to perform self-learning under constraints. By constructing the task logic, a multi-layer neural network is used to extract image features and fit more complex image relationships. Through preprocessing 2D images by deep learning, the image quality can be greatly optimized. By capturing different feature information on the image, different features on the image can be accurately processed and analyzed. However, although 3D reconstruction by deep learning generally improves the quality of 3D medical images, in the case of sparse views, there are still problems with difficult-to-remove complex artifacts and noise. The discreteness of 2D medical images will also inevitably have context matching problems, which leads to the loss of a lot of information that is difficult to recover during the reconstruction process. In addition, the generalization of the model is also one of the problems.

[0006] Generally speaking, although most of the current 3D reconstruction algorithms for the eye cup-disc can already complete the task of 3D visualization well, in the face of the problem of missing information, they still cannot accurately remove noise and artifacts. In addition, context matching is also an important problem. Considering the tissue lesions and individual differences at the same time, the versatility and transferability of the algorithm are also at a disadvantage. Facing the increasing clinical requirements, the current 3D reconstruction algorithms for the cup-disc still need further optimization and development. Summary of the Invention

[0007] In view of the deficiencies of the prior art, the present invention provides a cup-disc 3D reconstruction method and system based on distilled sampling and context matching. By optimizing the picture consistency based on the method of distilled sampling and context matching, the image optimization and 3D visualization of medical images are realized.

[0008] The first aspect of the present invention provides a cup-disc 3D reconstruction method based on distilled sampling and context matching, including the following specific steps:

[0009] Obtain 2D pictures of ocular medical images, and preprocess the obtained 2D pictures of ocular medical images, and then construct an image sequence and a mask image sequence; the image sequence includes several preprocessed 2D pictures of ocular medical images, and the mask image sequence includes mask images corresponding to each preprocessed 2D picture of ocular medical images;

[0010] Construct a generative model for gradually adding noise and gradually removing noise to the 2D pictures of ocular medical images to generate new 2D pictures of ocular medical images;

[0011] Use the image sequence to pre-train and fine-tune the generative model to obtain the final generative model;

[0012] Optimize the context matching of the image sequence using the final generation model to obtain an optimized image sequence. At the same time, optimize the mask image sequence using the final generation model to obtain an optimized mask image sequence;

[0013] Construct a NeRF model and train the NeRF model using the distillation sampling method to obtain a trained NeRF model;

[0014] Based on the optimized image sequence and the optimized mask image sequence, use the trained NeRF model to perform three-dimensional reconstruction of the cup and disc to obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc.

[0015] Further, obtain two-dimensional pictures of the ophthalmic medical images, preprocess the obtained two-dimensional pictures of the ophthalmic medical images, and then construct an image sequence and a mask image sequence. Specifically:

[0016] A1: Obtain several two-dimensional pictures of ophthalmic medical images, and each two-dimensional picture of the ophthalmic medical image corresponds to a section of the eye;

[0017] A2: Screen out the two-dimensional pictures of the ophthalmic medical images with cup and disc from the obtained two-dimensional pictures of the ophthalmic medical images; the cup and disc are the optic cup and the optic disc;

[0018] A3: Crop the screened two-dimensional pictures of the ophthalmic medical images according to a set size, and retain the central area of the two-dimensional pictures of the ophthalmic medical images to obtain the cropped two-dimensional pictures of the ophthalmic medical images;

[0019] A4: Determine the shape and position of the cup and disc in the cropped two-dimensional pictures of the ophthalmic medical images, and generate a mask image with the same size as the cropped two-dimensional pictures of the ophthalmic medical images according to the shape and position of the cup and disc. The area where the cup and disc are located is marked in the mask image;

[0020] A5: Normalize the cropped two-dimensional pictures of the ophthalmic medical images to obtain the normalized two-dimensional pictures of the ophthalmic medical images;

[0021] A6: Structurally store all the normalized two-dimensional pictures of the ophthalmic medical images and their corresponding mask images using sequences to obtain an image sequence CI and a mask image sequence M.

[0022] Furthermore, the generation model includes a noise scheduler and a UNet network. The noise scheduler is used to gradually add noise to the two-dimensional image of the ophthalmic medical image. The input of the UNet network is the two-dimensional image of the ophthalmic medical image after adding noise and the set internal and external camera parameters. The relative depth D is determined through the set internal and external camera parameters, and the two-dimensional image of the ophthalmic medical image after adding noise is gradually denoised according to the relative depth D to generate a new two-dimensional image of the ophthalmic medical image; the size of the new two-dimensional image of the ophthalmic medical image is the same as that of the input two-dimensional image of the ophthalmic medical image;

[0023] The relative depth is the relative depth between the generated new two-dimensional image of the ophthalmic medical image and the input two-dimensional image of the ophthalmic medical image;

[0024] The internal and external camera parameters include camera internal parameters and camera external parameters.

[0025] Furthermore, the generation model is pre-trained and fine-tuned using the image sequence to obtain the final generation model. Specifically:

[0026] B1: Divide the two-dimensional images of the ophthalmic medical images in the image sequence into a training set and a validation set according to a set ratio;

[0027] B2: Pre-train the generation model using the training set to obtain the pre-trained generation model;

[0028] B2.1: In the forward process, use the noise scheduler to gradually add noise to the two-dimensional image of the ophthalmic medical image to obtain the finally noise-added two-dimensional image of the ophthalmic medical image;

[0029] Specifically: At each time step t, Gaussian noise is added to the current two-dimensional image of the ophthalmic medical image and the weight of the two-dimensional image of the ophthalmic medical image t is determined through the hyperparameter a and the weight of the Gaussian noise such that the noise-added two-dimensional image of the ophthalmic medical image at each time step is:

[0030]

[0031] where x0 is the initial two-dimensional image of the ophthalmic medical image, and x t is the noise-added two-dimensional image of the ophthalmic medical image;

[0032] B2.2: In the backward process, use the UNet network to predict the noise and denoise the finally noise-added two-dimensional image of the ophthalmic medical image to obtain the generated two-dimensional image of the ophthalmic medical image;

[0033] Specifically: The UNet network learns the probability distribution p(x t-1 |x t ) and predicts the noise at the current time step Under the condition of time step t, subtract the predicted noise from the two-dimensional image x of the current noisy ophthalmic medical image t to obtain the two-dimensional image x of the denoised ophthalmic medical image ; t-1 ;

[0034] B2.3: When the time step t decreases, repeat B2.2 to denoise the two-dimensional image of the ophthalmic medical image until the time step t = 0, at which time the two-dimensional image of the generated ophthalmic medical image is obtained;

[0035] B2.4: Calculate the loss function and use the backpropagation algorithm to update the parameters of the generation model;

[0036] B2.5: Repeat steps B2.1 - B2.4 to obtain the trained generation model;

[0037] B3: Fine-tune the trained generation model using the validation set to obtain the final generation model;

[0038] B3.1: In the forward process, use the noise scheduler to add initial noise with a set distribution to the two-dimensional images of the ophthalmic medical images in the validation set;

[0039] B3.2: In the backward process, determine the relative depth D according to the input internal and external camera parameters, and use the UNet network to gradually denoise the two-dimensional images of the noisy ophthalmic medical images obtained in B3.1 according to the relative depth D until the time step t = 0 to obtain the two-dimensional images of the generated ophthalmic medical images;

[0040] B3.3: Calculate the loss function of the fine-tuning process and use the backpropagation algorithm to update the parameters of the generation model;

[0041] B3.4: Repeat steps B3.1 - B3.3 to obtain the final generation model.

[0042] Furthermore, use the final generation model to perform context matching optimization on the image sequence to obtain the optimized image sequence, and at the same time use the final generation model to optimize the mask image sequence to obtain the optimized mask image sequence. Specifically:

[0043] C1: Set the relative depth D, and use the UNet network in the final generation model to generate new two-dimensional images of the ophthalmic medical images according to the two-dimensional images of the ophthalmic medical images in the image sequence CI to form a new image sequence;

[0044] The image sequence CI is represented as {c1, c2, ……, c i , ……, c m}, 0 < i < m, where c i represents the two-dimensional picture of the i-th ophthalmic medical image, and m is the number of two-dimensional pictures of ophthalmic medical images;

[0045] Let the new two-dimensional picture of the ophthalmic medical image generated between c i and c i+1 be D is the relative depth between the newly generated two-dimensional picture of the ophthalmic medical image and the two-dimensional picture c i of the ophthalmic medical image;

[0046] Add the newly generated two-dimensional picture of the ophthalmic medical image to the image sequence CI, and the new image sequence is Repeat the process of generating new two-dimensional pictures of ophthalmic medical images, obtain several new two-dimensional pictures of ophthalmic medical images and add them to the image sequence CI, and then obtain the final new image sequence;

[0047] C2: Set the relative depth D = 0, input the two-dimensional pictures of the ophthalmic medical images in the image sequence CI into the UNet network in the final generation model to generate new two-dimensional pictures of the ophthalmic medical images, that is, the optimized two-dimensional pictures of the ophthalmic medical images;

[0048] C3: Replace the corresponding two-dimensional pictures of the ophthalmic medical images in the new image sequence with the optimized two-dimensional pictures of the ophthalmic medical images to obtain the optimized image sequence;

[0049] C4: Set the relative depth D = 0, input the mask images in the mask image sequence into the UNet network in the final generation model to generate new mask images, that is, the optimized mask images, and then obtain the optimized mask image sequence.

[0050] Furthermore, the NeRF model is constructed and the NeRF model is trained using the distilled sampling method to obtain the trained NeRF model, specifically:

[0051] D1: Construct and initialize the NeRF model;

[0052] D2: According to the internal and external camera parameters of the two-dimensional pictures of the ophthalmic medical images in the optimized image sequence, obtain the corresponding rendered image R in the initialized NeRF model;

[0053] D3: Add noise to the rendered image to obtain the noise-added rendered image;

[0054] D4: Set the relative depth D = 0, and use the UNet network in the trained generation network to denoise the noisy rendering image to generate a new rendering image;

[0055] D5: Calculate the loss function of the NeRF model;

[0056] D6: Update the parameters of the NeRF model according to the loss function of the NeRF model to obtain a trained NeRF model.

[0057] Furthermore, using the optimized image sequence and the optimized mask image sequence, perform three-dimensional reconstruction of the cup and disc using the trained NeRF model to obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc, specifically:

[0058] E1: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to perform background reconstruction to generate a three-dimensional model of the background;

[0059] Specifically: Use the optimized mask image sequence to determine the background and the internal and external camera parameters of the background in the optimized image sequence, and input the background and the internal and external camera parameters of the background into the trained NeRF model to generate a three-dimensional model of the background;

[0060] E2: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to perform three-dimensional reconstruction of the optic cup and optic disc to obtain a three-dimensional reconstruction model of the optic cup and optic disc;

[0061] Specifically: Use the optimized mask sequence to determine the optic cup and optic disc regions in the optimized image sequence, and input the optic cup and optic disc regions and the internal and external camera parameters into the trained NeRF model to obtain a three-dimensional reconstruction model of the optic cup and optic disc;

[0062] E3: Integrate the three-dimensional model of the background, the three-dimensional reconstruction model of the optic cup and optic disc, align them through a unified coordinate system, and optimize the visualization effect using rendering technology to generate a three-dimensional visualization model integrating the eye background, optic cup, and optic disc.

[0063] The second aspect of the present invention provides a three-dimensional reconstruction system for the cup and disc based on distilled sampling and context matching, which is used to implement a three-dimensional reconstruction method for the cup and disc based on distilled sampling and context matching, including:

[0064] A picture acquisition and sequence construction module, which is used to acquire two-dimensional pictures of eye medical images, preprocess the acquired two-dimensional pictures of eye medical images, and then construct an image sequence and a mask image sequence;

[0065] An optimization module for optimizing the context matching of an image sequence using a pre-trained and generative model to obtain an optimized image sequence, and simultaneously optimizing the mask image sequence using the final generative model to obtain an optimized mask image sequence;

[0066] A 3D reconstruction module for performing 3D reconstruction of the cup and disc using the trained NeRF model based on the optimized image sequence and the optimized mask image sequence to obtain a 3D visualization model integrating the background, optic cup, and optic disc.

[0067] A third aspect of the present invention provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the 3D reconstruction method of the cup and disc based on distillation sampling and context matching are executed.

[0068] A fourth aspect of the present invention provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of the 3D reconstruction method of the cup and disc based on distillation sampling and context matching are executed.

[0069] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0070] The present invention first realizes the standardization and structuring of medical images in the data preprocessing stage. By screening OCT images containing cup and disc structures, central cropping, normalization processing, and mask generation, the influence of equipment differences and individual differences on data quality is effectively eliminated, providing highly consistent inputs for subsequent deep learning. In addition, the present invention introduces a generative model combining a noise scheduler and a UNet network, uses the internal and external parameters of the camera to introduce relative depth calculation, and through a progressive noise addition and denoising mechanism, enables the model to accurately learn the feature distribution of medical images. The present invention also uses the method of distillation sampling to measure the consistency between two-dimensional images, and based on the idea of context matching, generates more images for further calibration of the context relationship, and then more accurately processes the relationship between two-dimensional images. In this way, more image information can be automatically extracted for subsequent 3D reconstruction. Such an effect is remarkable, which can not only remove the artifact problem between the reconstructed 3D images, but also improve the fineness of the 3D images to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a schematic diagram of the preprocessing process of the two-dimensional picture of the ophthalmic medical image provided by the embodiment of the present invention;

[0072] Figure 2Schematic diagrams of the forward and backward processes of the generation model provided by the embodiments of the present invention;

[0073] Figure 3 Schematic diagram of the Unet network architecture provided by the embodiments of the present invention;

[0074] Figure 4 Schematic diagram of the new image generation process provided by the embodiments of the present invention;

[0075] Figure 5 Schematic diagram of the optimized image process provided by the embodiments of the present invention;

[0076] Figure 6 Architecture diagram of the distilled sampling optimization model provided by the embodiments of the present invention. Detailed implementation manners

[0077] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0078] This embodiment provides a three-dimensional reconstruction method for cup and disc based on distilled sampling and context matching, including the following specific steps:

[0079] Step 1: Obtain two-dimensional pictures of ophthalmic medical images, and preprocess the obtained two-dimensional pictures of ophthalmic medical images, and then construct an image sequence and a mask image sequence; the image sequence includes several preprocessed two-dimensional pictures of ophthalmic medical images, and the mask image sequence includes mask images corresponding to each preprocessed two-dimensional picture of ophthalmic medical images;

[0080] The two-dimensional pictures of ophthalmic medical images are the information basis of the three-dimensional reconstruction algorithm. However, the two-dimensional pictures of ophthalmic medical images cannot be directly used for deep learning and need to be subjected to certain normalization processing;

[0081] As Figure 1 shown, specifically including:

[0082] Step 1.1: Obtain several two-dimensional pictures of ophthalmic medical images, and each two-dimensional picture of ophthalmic medical image corresponds to a section of the eye;

[0083] This example aims to preprocess the two-dimensional images of ophthalmic medical images in a standardized and normalized manner, providing consistent and high-quality input data for the training of subsequent deep learning models. Most of the sources of medical images are the examination data of clinical machines. First, OCT (Optical Coherence Tomography) images are extracted from clinical devices, and these images contain detailed information about the eye structure. Since OCT images may be affected by device differences, in the embodiments of the present invention, a script construction method is used to extract the OCT images (two-dimensional images of ophthalmic medical images) of the eyeball in a clinical fundus image capture instrument as training data, ensuring that the image acquisition method is unified and accurate. It should be noted that two-dimensional images of different types of ophthalmic medical images extracted by other methods are still applicable to the steps in this application;

[0084] Step 1.2: Screen out the two-dimensional images of ophthalmic medical images with cup and disc from the obtained two-dimensional images of ophthalmic medical images; the cup and disc refer to the optic cup and optic disc;

[0085] Next, in order to ensure that only OCT images containing the key eye structure - the cup and disc are used, OCT images with cup and disc parts are screened from the OCT images obtained according to the cup and disc positions of the vertical fundus images extracted from clinical devices, that is, if the projection of the current OCT image on the vertical fundus image intersects with the cup and disc, then there is a cup and disc part in this OCT image;

[0086] Step 1.3: Crop the screened two-dimensional images of ophthalmic medical images according to a set size, retain the central area of the two-dimensional images of ophthalmic medical images, and obtain the cropped two-dimensional images of ophthalmic medical images;

[0087] In this example, the screened OCT images also need to be cropped to unify the image size. The position of the cup and disc image is in the middle of the OCT image. Due to individual differences, the cropping operation ensures that the left and right sides of the image are cropped with the same number of pixels without affecting the upper and lower parts of the image, so as to crop the image into a square and retain the central area of the image;

[0088] Step 1.4: Determine the shape and position of the cup and disc in the cropped two-dimensional images of ophthalmic medical images, and generate a mask image with the same size as the cropped two-dimensional images of ophthalmic medical images according to the shape and position of the cup and disc, and the area where the cup and disc are located is marked in the mask image;

[0089] In this example, in order to further process these images, experts delimit or determine the shape and position of the cup and disc in the OCT image by other means, and generate a mask image with the same size as the OCT image in Step 1.3. The area where the cup and disc are located is marked in this mask image, providing important reference information for subsequent neural network processing;

[0090] Step 1.5: Normalize the two-dimensional image of the cropped ocular medical image to obtain the normalized two-dimensional image of the ocular medical image;

[0091] After the image mask is generated, in order to ensure that the OCT image meets the input requirements of the neural network, the OCT image must be normalized. In this embodiment, the maximum-minimum normalization method is used to scale the pixel values of the OCT image to the range of [0,1]. This not only ensures the consistency of the OCT image but also improves the stability of neural network training;

[0092] Step 1.6: Structurally store all the normalized two-dimensional images of the ocular medical images and their corresponding mask images using sequences respectively to obtain the image sequence CI and the mask image sequence M;

[0093] Finally, the processed images will be structurally stored. Specifically, the normalized OCT images and the corresponding mask images are stored in the image sequence CI and the mask image sequence M respectively, which is convenient for subsequent batch reading and neural network training. Through this series of preprocessing steps, the OCT images are standardized and normalized, providing high-quality inputs for subsequent deep learning applications and ensuring that the model can effectively learn on consistent data.

[0094] Step 2: Construct a generative model;

[0095] As Figure 2 shown, the generative model includes a noise scheduler and a UNet network. The noise scheduler is used to gradually add noise to the input two-dimensional image of the ocular medical image. The input of the UNet network is the two-dimensional image of the ocular medical image with added noise and the set internal and external camera parameters. The relative depth D is determined through the set internal and external camera parameters, and the two-dimensional image of the ocular medical image with added noise is gradually denoised according to the relative depth D to generate a new two-dimensional image of the ocular medical image; the size of the new two-dimensional image of the ocular medical image is the same as that of the input two-dimensional image of the ocular medical image;

[0096] The relative depth is the relative depth between the generated new two-dimensional image of the ocular medical image and the input two-dimensional image of the ocular medical image;

[0097] The internal and external camera parameters include camera internal parameters and camera external parameters;

[0098] As Figure 3 shown, the overall structure of the UNet network is divided into an input layer, an encoder, a decoder, and a skip connection module;

[0099] The input layer: The input is a two-dimensional image of an ophthalmic medical image and the internal and external camera parameters. The shape of the two-dimensional image of the ophthalmic medical image is H×W×C, representing the height, width, and number of channels of the two-dimensional image of the ophthalmic medical image respectively;

[0100] The encoder: It includes several sequentially connected convolutional blocks. The output of each convolutional block is the input of the next convolutional block, and pooling is performed once after every two convolutional blocks. The convolutional block includes a convolutional layer, a ReLU activation function, and a pooling layer;

[0101] The decoder: It includes several sequentially connected deconvolutional blocks. The output of each deconvolutional block is the input of the next deconvolutional block. The deconvolutional block includes a ReLU activation function, a convolutional layer, and a skip connection module;

[0102] The skip connection module: It directly connects the feature map output by the corresponding convolutional layer of the encoder with the feature map of the corresponding layer in the decoder. Specifically, the feature map output by a certain layer of the encoder is downsampled to obtain the feature map U, the corresponding layer of the decoder upsamples the feature map to obtain the feature map B, the obtained feature map U and the feature map B are concatenated along the channel dimension to obtain the connected feature map L, and the obtained feature map L is input into the corresponding part of the decoder;

[0103] Step 3: Use the image sequence to pre-train and fine-tune the generation model to obtain the final generation model;

[0104] Step 3.1: Divide the two-dimensional images of the ophthalmic medical images in the image sequence into a training set and a validation set according to a set ratio;

[0105] Step 3.2: Use the training set to pre-train the generation model to obtain the pre-trained generation model;

[0106] In this embodiment, a generation model with prior information is obtained through pre-training. The entire model consists of a forward process and a backward process. Therefore, appropriate key parameters and training processes need to be set for each process. In this embodiment, the time step of the forward process is randomly selected. However, to control the computational complexity, the value range of the time step is [2000, 6000]; half of the time step of the forward process is selected as the time step of the backward process, and hyperparameters such as the learning rate, Gaussian noise weight, and optimizer are set;

[0107] The process of pre-training the generation model specifically includes:

[0108] Step 3.2.1: In the forward process, use the noise scheduler to gradually add noise to the input two-dimensional image of the ophthalmic medical image to obtain the finally noise-added two-dimensional image of the ophthalmic medical image;

[0109] Specifically: At each time step t, Gaussian noise is added to the two-dimensional image of the current ophthalmic medical image. And through the hyperparameter a t Determine the weight of the two-dimensional image of the ophthalmic medical image And the Gaussian noise weight So that the two-dimensional image of the noise-added ophthalmic medical image at each time step is:

[0110]

[0111] Where, x0 is the two-dimensional image of the initial ophthalmic medical image, x t Is the two-dimensional image of the noise-added ophthalmic medical image, where the Gaussian noise The value of will continuously increase in intensity according to the training process;

[0112] Step 3.2.2: During the backward process, use the UNet network to predict the noise, denoise the two-dimensional image of the finally noise-added ophthalmic medical image, and obtain the generated two-dimensional image of the ophthalmic medical image;

[0113] Specifically: The UNet network learns the probability distribution p(x t-1 |x t ), predict the noise at the current time step Under the condition of time step t, subtract the predicted noise t From the current noise-added two-dimensional image x of the ophthalmic medical image To obtain the denoised two-dimensional image x of the ophthalmic medical image t-1 ;

[0114] Step 3.2.3: When the time step t decreases, repeat Step 3.2.2 to denoise the two-dimensional image of the ophthalmic medical image until the time step t = 0, at which time the generated two-dimensional image of the ophthalmic medical image is obtained;

[0115] Step 3.2.4: Calculate the loss function and use the backpropagation algorithm to update the parameters of the generation model;

[0116] Use the L2 loss function to calculate the difference between the noise predicted by the generation model And the real noise Introduce the time step t factor to optimize the parameters of the generation model. Specifically:

[0117]

[0118] Where, L is the loss function, Is the noise predicted at time step t, The real noise at time step t, and N is the number of time steps;

[0119] Step 3.2.5: Repeat steps 3.2.1 - 3.2.4 to obtain the trained generation model;

[0120] Step 3.3: Fine - tune the trained generation model using the validation set to obtain the final generation model;

[0121] Step 3.3.1: In the forward process, use the noise scheduler to add initial noise with a set distribution to the 2D images of the ophthalmic medical images in the validation set;

[0122] The initial noise in this embodiment follows a Gaussian distribution;

[0123] Step 3.3.2: In the backward process, determine the relative depth D according to the input camera internal and external parameters, and use the UNet network to gradually denoise the 2D images of the noisy ophthalmic medical images obtained in step 3.3.1 until the time step t = 0 to obtain the generated 2D images of the ophthalmic medical images;

[0124] Step 3.3.3: Calculate the loss function of the fine - tuning process and use the backpropagation algorithm to update the parameters of the generation model;

[0125] The loss function of the fine - tuning process is:

[0126]

[0127] where L ′ is the loss function of the fine - tuning process, min represents choosing the minimum value, θ represents the parameters of the generation model, E represents the expectation, z represents the noise - added variable, ε(x) represents the distribution followed by the noise - added variable, ∈ represents the standard Gaussian noise (following a Gaussian distribution), ∈ θ (z t ,t,f(x,D) represents the predicted noise, z t represents the noise - added variable at time step t, and f(x,D) represents the 2D images of the ophthalmic medical images generated according to the input 2D images x of the ophthalmic medical images and the relative depth D;

[0128] Step 3.3.4: Repeat steps 3.3.1 - 3.2.3 to obtain the final generation model;

[0129] Step 4: Use the final generation model to optimize the context matching of the image sequence to obtain the optimized image sequence, and at the same time use the final generation model to optimize the mask image sequence to obtain the optimized mask image sequence;

[0130] Based on the final generation model in this embodiment, on the one hand, generate more dense two-dimensional pictures of ophthalmic medical images for optimizing context matching, and on the other hand, optimize the current two-dimensional pictures of ophthalmic medical images to remove artifacts and noise, such as Figure 4 and Figure 5 shown, specifically including:

[0131] Step 4.1: Set the relative depth D, and use the UNet network in the final generation model to generate new two-dimensional pictures of ophthalmic medical images according to the two-dimensional pictures of ophthalmic medical images in the image sequence CI, forming a new image sequence;

[0132] The image sequence CI obtained in Step 1 is expressed as {c1, c2, ……, c i , ……, c m}, 0 < i < m, where c i represents the i-th two-dimensional picture of ophthalmic medical image, and m is the number of two-dimensional pictures of ophthalmic medical images;

[0133] The encoder ε performs high-dimensional representation on the two-dimensional pictures of ophthalmic medical images and the internal and external camera parameters in the input image sequence CI, and the decoder ∈ θ gradually denoises during the diffusion process. Through continuous optimization of the encoding and decoding process of noise by the encoder and decoder (i.e., the denoiser), and by setting conditions such as relative position, new two-dimensional pictures of ophthalmic medical images that meet the conditions are obtained;

[0134] Let the new two-dimensional picture of ophthalmic medical image generated between c i and c i+1 be D is the relative depth for generating the new two-dimensional picture of ophthalmic medical image and the two-dimensional picture c i of ophthalmic medical image, and the value of D is preset;

[0135] Add the generated new two-dimensional picture of ophthalmic medical image to the image sequence CI. The new image sequence is Repeat the process of generating new two-dimensional pictures of ophthalmic medical images, obtain several new two-dimensional pictures of ophthalmic medical images and add them to the image sequence CI, and then obtain the final new image sequence;

[0136] Step 4.2: Set the relative depth D = 0, input the two-dimensional pictures of ophthalmic medical images in the image sequence CI into the UNet network in the final generation model, and generate new two-dimensional pictures of ophthalmic medical images, that is, optimized two-dimensional pictures of ophthalmic medical images;

[0137] In this embodiment, the optimization of the two-dimensional image of the ophthalmic medical image is achieved by generating a new two-dimensional image of the ophthalmic medical image;

[0138] Step 4.3: Replace the two-dimensional image of the ophthalmic medical image corresponding in the new image sequence with the optimized two-dimensional image of the ophthalmic medical image to obtain an optimized image sequence;

[0139] Step 4.4: Set the relative depth D = 0, input the mask image in the mask image sequence into the UNet network in the final generation model to generate a new mask image, that is, an optimized mask image, and then obtain an optimized mask image sequence;

[0140] Step 5: Construct a NeRF model and train the NeRF model using the distillation sampling method to obtain a trained NeRF model, as Figure 6 shown;

[0141] In this step, based on the NeRF model, three-dimensional generation is performed using the distillation sampling method. The denoising process of the generation model is used to supervise the generation of the three-dimensional model, and the obtained three-dimensional visualization model exists in the form of NeRF (Neural Radiance Fields).

[0142] Step 5.1: Construct and initialize the NeRF model;

[0143] Use random parameters θ m Initialize a multi-layer perceptron (MLP) network to construct the NeRF model, which provides the initial network structure and parameters for the NeRF model;

[0144] Step 5.2: According to the internal and external camera parameters of the two-dimensional image of the ophthalmic medical image in the optimized image sequence in Step 4, obtain the corresponding rendering R in the initialized NeRF model. The resolution and other attributes (such as focal length, viewing angle, etc.) of the rendering are consistent with the two-dimensional image of the ophthalmic medical image in the optimized image sequence in Step 4;

[0145] Step 5.3: Add noise to the rendering to obtain a noisy rendering;

[0146] On the rendering obtained in Step 5.2, add noise δ, which follows a Gaussian distribution. The purpose of the noise is to simulate the uncertainty in the generation process for subsequent denoising operations;

[0147] Step 5.4: Set the relative depth D = 0, and use the UNet network in the trained generation network to denoise the noisy rendering to generate a new rendering;

[0148] With the help of the UNet network, noise prediction is performed, and the noise prediction result is denoted as wherein represents the estimation of noise by the UNet model. By calculating the probability distribution of noise by the UNet network, it can be obtained that UNet(z|t) represents the new rendered image output by the UNet network, where z is the noise-added variable;

[0149] Step 5.5: Calculate the loss function of the NeRF model;

[0150] According to the chain rule, construct the noise contrast term and substitute it into the gradient calculation of the NeRF model. The final loss function of the NeRF model is expressed as:

[0151]

[0152] where LN represents the loss function of the NeRF model, and k(t) represents the weighting function of the time step, is the gradient corresponding to the NeRF model;

[0153] Step 5.6: Update the parameters of the NeRF model according to the loss function of the NeRF model to obtain the trained NeRF model;

[0154] Use the standard optimizer and the loss function in Step 5.5 to optimize the NeRF model. Calculate the gradient of the loss function with respect to the NeRF model parameters θ m and update the network parameters to improve the quality of the generated three-dimensional images.

[0155] Step 6: According to the optimized image sequence and the optimized mask image sequence, use the trained NeRF model to perform three-dimensional reconstruction of the cup and disc to obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc;

[0156] In Step 5, the NeRF model has been optimized using the distilled sampling method and obtained. This model can reconstruct the three-dimensional structure from two-dimensional images. Based on this NeRF model, in this step, the reconstruction effect will be further improved through three-dimensional reconstruction, especially the modeling and visualization of the eye cup and disc.

[0157] Step 6.1: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to perform background reconstruction to generate a three-dimensional model of the background; the background includes other parts of the cut eyeball wall except the cup and disc, such as structures like the orbit and eyelid.

[0158] Specifically: Using the optimized mask image sequence, determine the background in the optimized image sequence and the internal and external camera parameters of the background. Input the background and the internal and external camera parameters of the background into the trained NeRF model. Based on the texture and geometric information of this background area, the NeRF model precisely models the details in the two-dimensional image, generates a three-dimensional model of the background, and obtains a complete three-dimensional background scene, ensuring that the spatial relationship between the eye and the background is real and accurate.

[0159] 6.2: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to perform three-dimensional reconstruction of the optic cup and optic disc, and obtain the three-dimensional reconstruction models of the optic cup and optic disc.

[0160] Specifically: Using the optimized mask sequence, determine the optic cup and optic disc areas in the optimized image sequence. Input the optic cup and optic disc areas and the internal and external camera parameters into the trained NeRF model for detailed three-dimensional reconstruction, paying special attention to the geometric shapes, depth information, and texture details of the optic cup and optic disc, ensuring the accurate reproduction of the structural features of these two areas, and obtaining fine three-dimensional reconstruction models of the optic cup and optic disc.

[0161] Step 6.3: Integrate the three-dimensional model of the background, the three-dimensional reconstruction models of the optic cup and optic disc, align them through a unified coordinate system to ensure the consistency of the spatial relationship between regions, and use rendering technology to optimize the visualization effect to ensure that the details of the entire eye area are clearly visible. Generate a complete three-dimensional visualization model integrating the eye background, optic cup, and optic disc.

[0162] This embodiment also provides a three-dimensional reconstruction system for the cup and disc based on distilled sampling and context matching, which is used to implement a three-dimensional reconstruction method for the cup and disc based on distilled sampling and context matching, including:

[0163] The picture acquisition and sequence construction module is used to acquire two-dimensional pictures of eye medical images, preprocess the acquired two-dimensional pictures of eye medical images, and then construct an image sequence and a mask image sequence.

[0164] The optimization module is used to perform context matching optimization on the image sequence using a pre-trained and generative model to obtain an optimized image sequence, and at the same time use the final generative model to optimize the mask image sequence to obtain an optimized mask image sequence.

[0165] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the cup and disc according to the optimized image sequence and the optimized mask image sequence, using the trained NeRF model, and obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc.

[0166] This embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the three-dimensional reconstruction method of cup and disc based on distillation sampling and context matching are executed.

[0167] This embodiment also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of the three-dimensional reconstruction method of cup and disc based on distillation sampling and context matching as described above are executed.

[0168] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present invention.

Claims

1. A three-dimensional reconstruction method of cup and plate based on distilled sampling and context matching, characterized in that It includes the following specific steps: Obtain two-dimensional images of ophthalmic medical images, and preprocess the obtained two-dimensional images of ophthalmic medical images, and then construct an image sequence and a mask image sequence; the image sequence includes several preprocessed two-dimensional images of ophthalmic medical images, and the mask image sequence includes mask images corresponding to each preprocessed two-dimensional image of ophthalmic medical images; Construct a generative model for gradually adding noise and gradually removing noise from two-dimensional images of ophthalmic medical images to generate new two-dimensional images of ophthalmic medical images; Use the image sequence to pre-train and fine-tune the generative model to obtain the final generative model; Use the final generative model to optimize the context matching of the image sequence to obtain an optimized image sequence, and at the same time use the final generative model to optimize the mask image sequence to obtain an optimized mask image sequence; Construct a NeRF model and use the distillation sampling method to train the NeRF model to obtain a trained NeRF model; According to the optimized image sequence and the optimized mask image sequence, use the trained NeRF model to perform three-dimensional reconstruction of the cup and disc to obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc.

2. The three-dimensional reconstruction method of cup and plate based on distillation sampling and context matching according to claim 1, characterized in that The obtaining of the two-dimensional images of ophthalmic medical images and the preprocessing of the obtained two-dimensional images of ophthalmic medical images, and then the construction of the image sequence and the mask image sequence are specifically as follows: A1: Obtain several two-dimensional images of ophthalmic medical images, and each two-dimensional image of ophthalmic medical image corresponds to a section of the eye; A2: Screen out the two-dimensional images of ophthalmic medical images with cups and discs from the obtained two-dimensional images of ophthalmic medical images; the cup and disc are the optic cup and the optic disc; A3: Crop the screened two-dimensional images of ophthalmic medical images according to a set size, and retain the central area of the two-dimensional images of ophthalmic medical images to obtain the cropped two-dimensional images of ophthalmic medical images; A4: Determine the shape and position of the cup and disc in the cropped two-dimensional images of ophthalmic medical images, and generate a mask image with the same size as the cropped two-dimensional images of ophthalmic medical images according to the shape and position of the cup and disc, and the area where the cup and disc are located is marked in the mask image; A5: Normalize the cropped two-dimensional images of ophthalmic medical images to obtain normalized two-dimensional images of ophthalmic medical images; A6: Structurally store all the normalized two-dimensional images of ophthalmic medical images and their corresponding mask images using sequences respectively to obtain an image sequence CI and a mask image sequence M.

3. The three-dimensional reconstruction method of cup and plate based on distillation sampling and context matching according to claim 1, characterized in that, The generative model includes a noise scheduler and a UNet network. The noise scheduler is used to gradually add noise to the input two-dimensional images of ophthalmic medical images. The UNet network takes the noise-added two-dimensional images of ophthalmic medical images and the set internal and external camera parameters as inputs, determines the relative depth D through the set internal and external camera parameters, and gradually removes noise from the noise-added two-dimensional images of ophthalmic medical images according to the relative depth D to generate new two-dimensional images of ophthalmic medical images; the new two-dimensional images of ophthalmic medical images are the same size as the input two-dimensional images of ophthalmic medical images; The relative depth is the relative depth between the two-dimensional picture of the newly generated ophthalmic medical image and the two-dimensional picture of the input ophthalmic medical image; The internal and external camera parameters include the internal camera parameters and the external camera parameters.

4. The three-dimensional reconstruction method of cup and plate based on distillation sampling and context matching according to claim 1, characterized in that, The method of using the image sequence to pre-train and fine-tune the generation model to obtain the final generation model is as follows: B1: Divide the two-dimensional pictures of the ophthalmic medical images in the image sequence into a training set and a validation set according to a set ratio; B2: Pre-train the generation model using the training set to obtain a pre-trained generation model; B2.1: In the forward process, gradually add noise to the two-dimensional picture of the input ophthalmic medical image using a noise scheduler to obtain the finally noise-added two-dimensional picture of the ophthalmic medical image; Specifically: At each time step t, Gaussian noise θ is added to the two-dimensional image of the current ophthalmic medical image, and the hyperparameter a t is used to determine the weights of the two-dimensional image of the ophthalmic medical image and the weights of the Gaussian noise such that the two-dimensional image of the noise-added ophthalmic medical image at each time step is: Among them, x0 is the two-dimensional image of the initial ophthalmic medical image, and x t is the two-dimensional image of the ophthalmic medical image with added noise; B2.2: In the backward process, use the UNet network to predict the noise and denoise the finally noise-added two-dimensional picture of the ophthalmic medical image to obtain the generated two-dimensional picture of the ophthalmic medical image; Specifically: The UNet network predicts the noise at the current time step by learning the probability distribution p(x t-1 |x t ). Under the condition of time step t, subtract the predicted noise from the two-dimensional image x of the current noisy ophthalmic medical image t to obtain the two-dimensional image x of the denoised ophthalmic medical image ; t-1 ​ B2.3: When the time step t decreases, repeat B2.2 to denoise the two-dimensional picture of the ophthalmic medical image until the time step t = 0, at which point the generated two-dimensional picture of the ophthalmic medical image is obtained; B2.4: Calculate the loss function and use the backpropagation algorithm to update the parameters of the generation model; B2.5: Repeat steps B2.1 - B2.4 to obtain a pre-trained generation model; B3: Fine-tune the pre-trained generation model using the validation set to obtain the final generation model; B3.1: In the forward process, use the noise scheduler to add initial noise with a set distribution to the two-dimensional pictures of the ophthalmic medical images in the validation set; B3.2: In the backward process, determine the relative depth D according to the input internal and external camera parameters, and use the UNet network in the final generation model to gradually denoise the noise-added two-dimensional picture of the ophthalmic medical image obtained in B3.1 until the time step t = 0 to obtain the generated two-dimensional picture of the ophthalmic medical image; B3.3: Calculate the loss function in the fine-tuning process and use the backpropagation algorithm to update the parameters of the generation model; B3.4: Repeat steps B3.1 - B3.3 to obtain the final generation model.

5. The three-dimensional reconstruction method of cup and plate based on distilled sampling and context matching according to claim 1, wherein The method of using the final generation model to perform context matching optimization on the image sequence to obtain an optimized image sequence, and at the same time using the final generation model to optimize the mask image sequence to obtain an optimized mask image sequence is as follows: C1: Set the relative depth D, and use the UNet network in the final generation model to generate a new two-dimensional picture of the ophthalmic medical image according to the two-dimensional pictures of the ophthalmic medical images in the image sequence CI to form a new image sequence; The image sequence CI is represented as {c1, c2, ……, c i , ……, c m}, 0 < i < m, where c i represents the two-dimensional picture of the i-th ophthalmic medical image, and m is the number of two-dimensional pictures of ophthalmic medical images; Let c i and c i+1 generate a two-dimensional image of a new ophthalmic medical image between them as Let D be the two-dimensional image of the newly generated ophthalmic medical image and the relative depth between the two-dimensional image c of the ophthalmic medical image i is; The two-dimensional picture of the newly generated ophthalmic medical image is added to the image sequence CI, and the new image sequence is The process of generating the two-dimensional picture of the newly generated ophthalmic medical image is repeated to obtain several two-dimensional pictures of the newly generated ophthalmic medical image and added to the image sequence CI, thereby obtaining the final new image sequence; C2: Set the relative depth D = 0, input the two-dimensional pictures of the ophthalmic medical images in the image sequence CI into the UNet network in the final generation model to generate a new two-dimensional picture of the ophthalmic medical image, that is, the optimized two-dimensional picture of the ophthalmic medical image; C3: Replace the corresponding two-dimensional pictures of the ophthalmic medical images in the new image sequence with the optimized two-dimensional pictures of the ophthalmic medical images to obtain the optimized image sequence; C4: Set the relative depth D = 0, input the mask images in the mask image sequence into the UNet network in the final generation model to generate new mask images, that is, optimized mask images, and then obtain an optimized mask image sequence.

6. The three-dimensional reconstruction method of cup and plate based on distillation sampling and context matching according to claim 1, characterized in that The construction of the NeRF model and the training of the NeRF model using the distilled sampling method to obtain a trained NeRF model are specifically as follows: D1: Construct and initialize the NeRF model; D2: According to the internal and external camera parameters of the two-dimensional pictures of the ophthalmic medical images in the optimized image sequence, obtain the corresponding rendered image R in the initialized NeRF model; D3: Add noise to the rendered image to obtain a noisy rendered image; D4: Set the relative depth D = 0, and use the UNet network in the trained generation network to denoise the noisy rendered image to generate a new rendered image; D6: Calculate the loss function of the NeRF model; D7: Update the parameters of the NeRF model according to the loss function of the NeRF model to obtain a trained NeRF model.

7. The three-dimensional reconstruction method of cup and plate based on distillation sampling and context matching according to claim 1, characterized in that The three-dimensional reconstruction of the cup and disc using the trained NeRF model according to the optimized image sequence and the optimized mask image sequence to obtain a three-dimensional visualization model integrating the background, optic cup, and optic disc is specifically as follows: E1: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to reconstruct the background and generate a three-dimensional model of the background; Specifically: Use the optimized mask image sequence to determine the background and the internal and external camera parameters of the background in the optimized image sequence, and input the background and the internal and external camera parameters of the background into the trained NeRF model to generate a three-dimensional model of the background; E2: According to the optimized image sequence and the optimized image mask sequence, use the trained NeRF model to perform three-dimensional reconstruction of the optic cup and optic disc to obtain a three-dimensional reconstruction model of the optic cup and optic disc; Specifically: Use the optimized mask sequence to determine the optic cup and optic disc regions in the optimized image sequence, and input the optic cup and optic disc regions and the internal and external camera parameters into the trained NeRF model to obtain a three-dimensional reconstruction model of the optic cup and optic disc; E3: Integrate the three-dimensional model of the background, the three-dimensional reconstruction model of the optic cup and optic disc, align them through a unified coordinate system, and optimize the visualization effect using rendering technology to generate a three-dimensional visualization model integrating the ocular background, optic cup, and optic disc.

8. A three-dimensional reconstruction system of cup and disc based on distilled sampling and context matching, which is used to implement a three-dimensional reconstruction method of cup and disc based on distilled sampling and context matching according to any one of claims 1-7, characterized in that, Including: A picture acquisition and sequence construction module, which is used to acquire two-dimensional pictures of ophthalmic medical images, preprocess the acquired two-dimensional pictures of ophthalmic medical images, and then construct an image sequence and a mask image sequence; An optimization module, which is used to optimize the image sequence through context matching using a pre-trained and generation model to obtain an optimized image sequence, and at the same time optimize the mask image sequence using the final generation model to obtain an optimized mask image sequence; A three-dimensional reconstruction module, which is used to perform three-dimensional reconstruction of the cup and disc using the trained NeRF model according to the optimized image sequence and the optimized mask image sequence, so as to obtain a three-dimensional visualization model integrating the background, optic cup and optic disc.

9. An electronic device, characterized in that, It includes: A processor, a memory and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of a three-dimensional reconstruction method of a cup and disc based on distillation sampling and context matching according to any one of claims 1-7 are executed.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of a three-dimensional reconstruction method of a cup and disc based on distillation sampling and context matching according to claims 1-7 are executed.

Images