Neural radiance field volume super-resolution reconstruction method for cerebral microbleed detection

Abstract

The application discloses a kind of neural radiation field volume super-resolution reconstruction methods for brain microhemorrhage detection.First, the low-resolution SWI body data of patient is acquired, sample is constructed by multi-scale coordinate sampling, including: local sparse coordinate sampling is carried out on slice to obtain the first-level coordinate set of hollow patch, then three-dimensional radial coordinate sampling is carried out to each coordinate to obtain the second-level coordinate set;Subsequently, a neural radiation field model is constructed, the sample is input into the model and combined with the spherical body rendering technique to generate the final rendering value;Then, according to the preset super-resolution multiple a interval selection slice constitutes training set and to be reconstructed set, after training the model, the to be reconstructed set is predicted, and finally the high-resolution SWI body data with the number of Z-axis slices increased is obtained.The application does not need paired training data, and the local structure reconstruction fidelity is significantly improved by the innovative sampling strategy, and the unique magnetic susceptibility characteristics of SWI can be accurately reconstructed, which effectively overcomes the shortcomings of the prior art.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of medical image processing and computer vision, and in particular to a method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds. Background Technology

[0002] Susceptibility-weighted imaging (SWI) can sensitively detect brain microbleeds (CMB), but its image quality is limited by magnetic field strength and scan time. To reduce costs or motion artifacts, clinical practice often uses large slice thickness acquisition protocols, which leads to decreased image resolution and significant anisotropy, blurring of coronal and sagittal structures, severely restricting lesion interpretation and automatic detection performance.

[0003] For super-resolution reconstruction of low-resolution SWI images, existing techniques have evolved from interpolation methods to deep learning. In recent years, implicit neural representations (INRs) have achieved reconstruction at arbitrary scales, but their global modeling struggles to capture the complex local details of medical images. Although some studies have introduced convolutional neural networks to enhance details, these methods rely on large amounts of paired data for training, and the low-resolution images synthesized based on simple downsampling cannot simulate artifacts and noise in real clinical acquisitions, resulting in insufficient generalization ability of the models in real-world scenarios.

[0004] Neural Radiance Fields (NeRF), an important branch of INR, achieves high-quality reconstruction of complex 3D scenes by jointly modeling the volume density and color information of spatial coordinates and combining position encoding and hierarchical sampling. Based on this, CuNeRF et al. introduced the NeRF framework into the task of volumetric super-resolution reconstruction of medical images, achieving zero-sample, arbitrary-scale super-resolution reconstruction on a single low-resolution input, reducing the dependence on paired data. However, existing NeRF-based methods still have significant limitations in medical image, especially SWI volumetric data reconstruction: First, their supervision method based on random ray or cube sampling ignores the spatial correlation between pixels / voxels and lacks the ability to model the overall structure of image blocks, easily leading to overly smooth reconstruction results and loss of local details; Second, the standard NeRF sampling strategy fails to fully consider the radial signal variation characteristics caused by differences in magnetic susceptibility in SWI images, resulting in poor reconstruction of magnetic susceptibility artifacts such as low-signal "halos" common around CMB, which are crucial for the diagnosis of CMB.

[0005] In summary, existing super-resolution reconstruction methods still face three major challenges when applied to CMB detection:

[0006] (1) Over-reliance on paired high- and low-resolution training data;

[0007] (2) Insufficient ability to preserve local structure in SWI volumetric data;

[0008] (3) The unique magnetic susceptibility physical imaging characteristics of SWI were not fully considered, resulting in unsatisfactory reconstruction of CMB-related fine structures. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a super-resolution reconstruction method for the volume of neural radiation field for the detection of cerebral microbleeds. This method can significantly improve the reconstruction fidelity of local structures without the need for paired high- and low-resolution training data, and accurately reconstruct the magnetic susceptibility characteristics unique to SWI (magnetic susceptibility-weighted imaging), thereby overcoming the shortcomings of the prior art.

[0010] The technical solution adopted by this invention to solve the above-mentioned technical problems is as follows: a method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds, comprising the following steps:

[0011] S1. Obtain low-resolution SWI volume data from a patient, and perform multi-scale coordinate sampling on the slices, including:

[0012] Local sparse coordinate sampling: Multiple different hole patches are repeatedly sampled on each slice, and each hole patch provides a first-level coordinate set containing the center pixel and its sparsely distributed surrounding pixels;

[0013] Three-dimensional radial coordinate sampling: For each coordinate in each primary coordinate set, multiple first sampling points are sampled in a three-dimensional spherical space with a preset radius and centered on it, and their Cartesian coordinates are used to form a secondary coordinate set;

[0014] The set of all secondary coordinates corresponding to a single hole patch is aggregated into a single sample;

[0015] S2. Construct a neural radiation field model, in which a multilayer perceptron is used to learn the mapping from Cartesian coordinates to color values ​​and volume density;

[0016] Each set of secondary coordinates in the sample is input into the model, and combined with spherical rendering technology, the final rendering value of the corresponding pixel is generated and output.

[0017] S3. According to the preset Z-axis super-resolution factor a, select one slice from the low-resolution SWI volume data every a-1 frames. The samples corresponding to all selected slices constitute the training set, and the samples corresponding to all remaining slices constitute the set to be reconstructed.

[0018] S4. Train the model based on the training set to obtain a trained model;

[0019] S5. The trained model is used to predict the set to be reconstructed. Based on the final output rendering value, the reconstructed image block corresponding to the hole patch is generated, thereby obtaining high-resolution SWI volume data with an increased number of Z-axis slices.

[0020] In one possible embodiment, the sampling process for the hole patch in step S1 is as follows: A pixel is randomly selected as the center pixel in the slice, and around the center pixel, a hole patch of size ((h-1)d+1)×((h-1)d+1) is sampled according to a preset patch size factor h and a preset hole coefficient d. This invention, by limiting the sampling of hole patches, transforms the innovative idea of ​​local sparse coordinate sampling into a clear, quantifiable, and repeatable specific operational scheme.

[0021] In one possible embodiment, the process of sampling a first sampling point in the three-dimensional spherical space in step S1 is as follows: Based on three independent random variables that all follow a uniform distribution in [0,1], the radial distance, azimuth angle, and polar angle are calculated respectively to determine a first sampling point. This invention ensures that the three-dimensional radial coordinate sampling can accurately reflect the spherically symmetric physical field distribution, avoiding model learning bias caused by uneven sampling, and laying a solid foundation for subsequent high-fidelity volume rendering.

[0022] In one possible implementation, the radial distance r = R × u 1 / 3 The azimuth angle θ = 2π × δ, the polar angle φ = arccos(2η-1), where u, δ, and η are three random variables, and R represents the preset radius.

[0023] In one possible implementation, the specific process of inputting each set of secondary coordinates in the sample into the model and generating the final rendered value of the corresponding pixel using spherical rendering technology in step S2 is as follows:

[0024] S21. Input each set of secondary coordinates in the sample into the first multilayer perceptron in the model to obtain the first color value and the first volume density of the first sampling point;

[0025] S22. Based on the first volume density, sample multiple second sampling points within the same three-dimensional spherical space;

[0026] S23. Input the Cartesian coordinates of the two types of sampling points into the second multilayer perceptron in the model to obtain the second color value and second volume density of the corresponding sampling points, and generate the final rendering value of the corresponding pixels of all sampling points through spherical volume rendering technology.

[0027] In one possible implementation, the sampling process for the second sampling point in step S22 is as follows:

[0028] S221. Based on the first volume density of all first sampling points, calculate the first weighted value ω for each of the first sampling points. i,1 =T i,1 (1-exp(-σ i,1 (r i+1 -r i ))), where i=1,2,…,N c N c ω represents the number of the first sampling points. i,1 T represents the first weighted value of the i-th first sampling point. i,1 This represents the first transmittance of the i-th first sampling point. , σ i,1 Let r represent the first volume density of the i-th first sampling point. i This represents the radial distance of the i-th first sampling point within its three-dimensional spherical space, where i = 1, 2, ..., N. c -1 hour r i+1 This represents the radial distance of the (i+1)th first sampling point within its three-dimensional spherical space, where i=N c time r i+1 =r i ;

[0029] S222. Arrange the first weighted values ​​of all first sampling points in descending order, and take the radial distance of the first sampling points corresponding to the first weighted values ​​in their respective three-dimensional spherical spaces.

[0030] S223. Based on the two independent random variables introduced, both of which follow a uniform distribution in [0,1], calculate the azimuth and polar angle respectively, and determine a second sampling point by combining the radial distance obtained in step S222.

[0031] S224. Repeat step S223 to sample multiple second sampling points at different radial distances.

[0032] In one possible implementation, the formula for generating the final rendered value of the pixel corresponding to all sampling points in step S23 using spherical rendering technology is as follows: , where P z r represents the final rendered value of the pixel corresponding to all sampling points. j ω represents the radial distance of the j-th sampling point within its three-dimensional spherical space. j,2 ω represents the second weighted value of the j-th sampling point. j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2This represents the second transmittance at the j-th sampling point. , σ j,2 Let j represent the second volume density at the j-th sampling point, j=1,2,…,N c +N f -1 hour r j+1 This represents the radial distance of the (j+1)th sampling point within its three-dimensional spherical space, where j=N c +N f time r j+1 =r j , σ k,2 c represents the second volume density at the k-th sampling point. j,2 This represents the second color value of the j-th sampling point.

[0033] In one possible implementation, the model is trained with an image patch consisting of all pixels in the hole patch as the ground truth, and the network loss is calculated after each training round, which is a weighted sum of adaptive multi-scale structural similarity loss and adaptive mean squared error loss.

[0034] In one possible implementation embodiment, the adaptive multi-scale structural similarity loss is calculated as follows: , where L AM-SSIM AM-SSIM(B) represents the adaptive multi-scale structural similarity loss, where Num represents the batch size for each training round. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t Multiscale structural similarity between them M represents the number of scales in the multiscale decomposition, bri M (B n,pred B n,g.t. ), con m (B n,pred B n,g.t. ), str m (B n,pred B n,g.t. ) corresponds to B n,pred With B n,g.t Structural similarity among luminance, contrast, and structural components at corresponding scales, α M β m τ m The corresponding learned weight parameters represent the brightness, contrast, and structure components at the corresponding scale.

[0035] In one possible implementation embodiment, the adaptive mean squared error loss is calculated as follows: , where L A-MSEA-MSE(B) represents the adaptive mean squared error loss. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The mean square error between them B represents B n,pred With B n,g.t The number of pixels in P b,g.t. B n,g.t The pixel value of the b-th pixel in the image, P b,z B n,pred The pixel value of the b-th pixel in the image, P b,s Indicates with B n,g.t Corresponding initial rendered image block B n,s,pred The pixel value of the b-th pixel in the image is given. The initial rendered image block is composed of the initial rendered values ​​of all pixels in the hole patch corresponding to a sample in the training set.

[0036] In model training, this invention introduces a composite loss function that combines adaptive multi-scale structural similarity loss and adaptive mean squared error loss. Through joint supervision of the initial and final rendering, it effectively alleviates overfitting, ensures robust model optimization, and enhances the model's ability to perceive and reconstruct local organizational structures.

[0037] Compared with the prior art, the advantages of the present invention are as follows:

[0038] This invention achieves zero-sample, arbitrary-scale super-resolution reconstruction without the need for paired data. By selecting one slice every a-1 frames from low-resolution SWI volumetric data according to a preset Z-axis super-resolution factor 'a', this invention constructs a training set and a reconstruction set, utilizing the spatial continuity of the data itself to achieve self-supervised learning. This invention eliminates the need for any external paired high- and low-resolution training data, fundamentally solving the problem of excessive reliance on paired data in existing methods. Furthermore, because training and prediction are performed within the same data domain, it significantly improves the generalization ability for reconstructing real low-resolution clinical data.

[0039] This invention significantly improves the fidelity of local structures through multi-scale coordinate sampling. The local sparse coordinate sampling strategy employed in this invention acquires a first-level coordinate set containing the central pixel and its sparsely distributed surrounding pixels, enabling the model to perceive local neighborhood context information and improving the accuracy of reconstructing fine local structures. This abandons the isolated pixel sampling method in traditional NeRF, allowing the final reconstructed rendering values ​​to accurately preserve the morphology and structure of subtle lesions such as brain microhemorrhages, effectively avoiding over-smoothing and generating reconstructed image patches and high-resolution SWI volume data with greater diagnostic value.

[0040] This invention accurately reconstructs magnetic susceptibility characteristics through a physically driven sampling and rendering mechanism. The three-dimensional radial coordinate sampling and spherical rendering technology introduced in this invention samples multiple first sampling points within a three-dimensional spherical space at each coordinate and renders them, adapting the general neural radiation field from a visually rendered representation to a reconstructed physical field signal. This allows the constructed neural radiation field model to specifically learn and accurately generate the radial signal distribution caused by differences in magnetic susceptibility, thus clearly presenting crucial low-signal "halo" features around brain microbleeds in the final reconstruction result.

[0041] Through the aforementioned synergistic effect, high-resolution SWI volume data with an increased number of Z-axis slices was finally reconstructed, providing a reliable imaging tool for the accurate detection of cerebral microbleeds.

[0042] This invention achieves flexible, arbitrary-scale volumetric super-resolution by presetting a super-resolution factor 'a' to increase the resolution by any factor in the Z-axis direction.

[0043] This invention is applicable to the reconstruction of microscopic magnetically sensitive structures, such as cerebral microbleeds (CMBs) and vascular networks, thereby achieving high-quality super-resolution reconstruction of low-resolution SWI volume data. Extensive experiments on SWI datasets targeting cerebral microbleeds demonstrate that this invention consistently outperforms existing methods in arbitrary-scale super-resolution reconstruction tasks, particularly excelling in preserving tissue texture and structure, while exhibiting good robustness and high accuracy in CMB lesion reconstruction. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a block diagram illustrating the implementation of the method of the present invention;

[0046] Figure 2 A schematic diagram of a hole patch and multiple first sampling points on a slice;

[0047] Figure 3 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the LHL-SWI-CMB dataset;

[0048] Figure 4The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the HKCU-SWI-CMB dataset;

[0049] Figure 5 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the MSD-T1 dataset;

[0050] Figure 6 A statistical comparison diagram of segmentation performance for different super-resolution test datasets. Detailed Implementation

[0051] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0052] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0053] Existing super-resolution reconstruction methods have the following technical problems when applied to CMB detection: over-reliance on paired high- and low-resolution training data; insufficient ability to preserve local structures in SWI volume data; and failure to fully consider the unique magnetic susceptibility physical imaging characteristics of SWI, resulting in unsatisfactory reconstruction results of CMB-related fine structures.

[0054] To address the problems of existing technologies, this invention provides a method for super-resolution reconstruction of neural radiation field volume for detecting brain microbleeds. This method can significantly improve the reconstruction fidelity of local structures and accurately reconstruct the magnetic susceptibility characteristics unique to SWI without requiring paired high- and low-resolution training data.

[0055] The following is a detailed description of a neural radiation field volume super-resolution reconstruction method for detecting cerebral microbleeds provided by an embodiment of the present invention.

[0056] like Figure 1 As shown, the method provided in this embodiment of the invention includes the following steps:

[0057] S1. Obtain low-resolution SWI volume data from a patient, and perform multi-scale coordinate sampling on the slices contained therein to construct a sample. The multi-scale coordinate sampling includes:

[0058] Local sparse coordinate sampling: Multiple different hole patches are repeatedly sampled on each slice, and each hole patch provides a first-level coordinate set containing the center pixel and its sparsely distributed surrounding pixels.

[0059] Three-dimensional radial coordinate sampling: For each coordinate in each primary coordinate set, multiple first sampling points are sampled in a three-dimensional spherical space with a preset radius and centered on it, and their Cartesian coordinates are used to form a secondary coordinate set.

[0060] All secondary coordinates corresponding to a hole patch are aggregated into a single sample.

[0061] Figure 2 This shows a hole patch on the slice and multiple first sampling points.

[0062] Here, low-resolution SWI volume data refers to SWI volume data with a small number of slices in the Z-axis direction.

[0063] In some embodiments, low-resolution SWI volume data are acquired from clinical magnetic resonance imaging equipment, which typically has a thicker slice thickness, i.e., a lower inter-slice resolution.

[0064] In some embodiments, after obtaining low-resolution SWI volume data, de-identification and normalization processes are performed first to ensure data privacy and strength consistency.

[0065] In some embodiments, the process of sampling a hole patch on a slice is as follows: randomly select a pixel in the slice as the center pixel, and around the center pixel, according to the preset patch size factor h and the preset hole coefficient d, sample a hole patch of size ((h-1)d+1)×((h-1)d+1).

[0066] In some embodiments, the sampling of hole patches is random, repeated, and ensures that all sampled hole patches cover as many pixels as possible within the slice. This hole patch sampling strategy replaces the isolated and independent pixel sampling in NeRF and CuNeRF.

[0067] Here, both the patch size factor h and the hole coefficient d are preset values. The hole coefficient d is used to control the spacing between adjacent pixels. As an example, we take d=2 and h=3. In this case, there is a hole between adjacent pixels, the size of the sampled hole patch is 5×5, and the first-level coordinate set contains the coordinates of the center pixel of the hole patch and the coordinates of the 8 sparsely distributed pixels around it, that is, it contains the coordinates of a total of 9 pixels.

[0068] As an example, when a hole patch contains 9 pixels, the corresponding first-level coordinate set contains the coordinates of these 9 pixels. Taking the coordinates of the 5th pixel as an example, N is sampled within a three-dimensional spherical space of a preset radius with the 5th pixel as the center. c There are 9 first sampling points, and the Cartesian coordinates of these first sampling points are used to form a second-level coordinate set. These 9 second-level coordinate sets are then aggregated into a single sample.

[0069] In some embodiments, the process of sampling a first sampling point in a three-dimensional spherical space is as follows: three independent random variables are introduced, each of which follows a uniform distribution of [0,1]; then the radial distance, azimuth angle and polar angle in spherical coordinates are calculated based on the three random variables respectively, thereby determining a first sampling point.

[0070] As an example, if N samples are taken in a three-dimensional spherical space c If there is a first sampling point, then repeat the sampling process for the first sampling point N. c Next, due to the introduction of random variables, therefore N c Each of the first sampling points is different. To meet the sampling density requirements, in this embodiment, N samples are taken within a three-dimensional spherical space. c =64 first sampling points. This sampling strategy within a three-dimensional spherical space is consistent with the radial distribution characteristics of the magnetic susceptibility signal in SWI. Sampling on a concentric sphere achieves comprehensive spatial coverage of the surrounding radial region, effectively capturing circular or annular structural patterns, such as the low-signal halos commonly found in CMB.

[0071] In some embodiments, the radial distance r = R × u 1 / 3The azimuth angle θ = 2π × δ, and the polar angle φ = arccos(2η-1), where u, δ, and η are three random variables, and R represents the preset radius. Here, r is used to control the distance from the sampling point to the center of the sphere, r ∈ [0, R]; θ is used to specify the horizontal direction of the sampling point, θ ∈ [0, 2π]; φ is used to specify the vertical direction of the sampling point, φ ∈ [0, π]; R is a hyperparameter, which is set to R = 1 / 2 in this embodiment.

[0072] In some embodiments, the method further includes converting the spherical coordinates of the first sampling point to Cartesian coordinates, while converting the spherical coordinates of a point to Cartesian coordinates is a conventional technique.

[0073] S2. Construct a neural radiation field model, the core of which is a multilayer perceptron, to learn the mapping from Cartesian coordinates to color values ​​and volume density.

[0074] Each set of secondary coordinates in the sample is input into the model, and combined with spherical rendering technology, the final rendered value of the corresponding pixel is generated and output.

[0075] In some embodiments, the neural radiation field model includes a first multilayer perceptron and a second multilayer perceptron. Here, the number of layers in the multilayer perceptron can be designed to be 6-9, and in a specific implementation, the number of layers is designed to be 8. The specific process of inputting each set of secondary coordinates in the sample into the model and combining it with spherical rendering technology to generate the final rendered value of the corresponding pixel is as follows:

[0076] S21. Input each set of secondary coordinates in the sample into the first multilayer perceptron in the model to obtain the first color value and the first volume density of the first sampling point.

[0077] As an example, a set of two-dimensional coordinates contains N c N Cartesian coordinates, the first multilayer perceptron outputs N. c The first color value and the first volume density of the first sampling point.

[0078] S22. Based on the first volume density, continue to sample multiple second sampling points within the same three-dimensional spherical space.

[0079] In some embodiments, the sampling process for the second sampling point is as follows:

[0080] S221. Based on the first volume density of all first sampling points, calculate the first weighted value of each of the first sampling points. The calculation formula is: ω i,1 =T i,1 (1-exp(-σ i,1 (r i+1 -r i ))), where i=1,2,…,N c N cω represents the number of the first sampling points. i,1 T represents the first weighted value of the i-th first sampling point. i,1 This represents the first transmittance of the i-th first sampling point. exp represents an exponential function with base e, σ i,1 Let r represent the first volume density of the i-th first sampling point. i This represents the radial distance of the i-th first sampling point within its three-dimensional spherical space, where i = 1, 2, ..., N. c -1 hour r i+1 This represents the radial distance of the (i+1)th first sampling point within its three-dimensional spherical space, where i=N c time r i+1 =r i , here r k σ is the radial distance of the k-th first sampling point within its three-dimensional spherical space. k,1 Let r represent the first volume density of the k-th first sampling point. k This represents the radial distance of the k-th first sampling point within its three-dimensional spherical space, k=1,2,…,N c -1 hour r k+1 This represents the radial distance of the (k+1)th first sampling point within its three-dimensional spherical space, where k=N. c time r k+1 =r k .

[0081] S222. Arrange the first weighted values ​​of all first sampling points in descending order, and take the radial distance of the first sampling points corresponding to the first few first weighted values ​​in their respective three-dimensional spherical spaces.

[0082] As an example, for N c The first weighted values ​​of the first sampling points are arranged in descending order, and the first 3 first weighted values ​​are taken. It is assumed that the radial distances of the first sampling points corresponding to these first weighted values ​​in their respective three-dimensional spherical spaces are r1, r2, and r3.

[0083] S223. Introduce two independent random variables that both follow a uniform distribution in [0,1]. Then, calculate the azimuth and polar angle in spherical coordinates based on the two random variables, and determine a second sampling point by combining the radial distance obtained in step S222.

[0084] S224. Repeat step S223 to sample multiple second sampling points at different radial distances.

[0085] As an example, a second sampling point is obtained by taking a radial distance r1 and combining it with the azimuth and polar angles calculated from two random variables. At this radial distance r1, multiple random variables are taken to obtain multiple second sampling points with the same radial distance but different azimuth and polar angles. Similarly, multiple second sampling points with the same radial distance but different azimuth and polar angles are obtained at radial distances r2 and r3, respectively.

[0086] As an example, based on N c =The first volume density of the first sampling points is obtained by sampling N. f =64 second sampling points.

[0087] S23. Input the Cartesian coordinates of the two types of sampling points into the second multilayer perceptron in the model to obtain the second color value and second volume density of the corresponding sampling points. Then, through spherical volume rendering technology, generate the final rendering value of the pixels corresponding to all sampling points and output it.

[0088] In some embodiments, the formula for generating the final rendered value of the pixel corresponding to all sampling points using spherical volume rendering technology is as follows: , where P z r represents the final rendered value of the pixel corresponding to all sampling points. j ω represents the radial distance of the j-th sampling point (which may be the first or second sampling point) within its three-dimensional spherical space. j,2 ω represents the second weighted value of the j-th sampling point. j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2 This represents the second transmittance at the j-th sampling point. , σ j,2 Let j represent the second volume density at the j-th sampling point, j=1,2,…,N c +N f -1 hour r j+1 This represents the radial distance of the (j+1)th sampling point within its three-dimensional spherical space, where j=N c +N f time r j+1 =r j , σ k,2 c represents the second volume density at the k-th sampling point. j,2 This represents the second color value of the j-th sampling point.

[0089] As an example, there are N sampling points of two types. c =64 first sampling points and N f=64 second sampling points, and through spherical volume rendering technology, the final rendering value of one pixel corresponding to these 128 sampling points is generated.

[0090] S3. According to the preset Z-axis super-resolution factor a, select one slice from the low-resolution SWI volume data at intervals of a-1. The samples corresponding to all selected slices constitute the training set, and the samples corresponding to all remaining slices constitute the set to be reconstructed.

[0091] In this embodiment, a = 2 or 3.

[0092] S4. Train the model based on the training set to obtain a trained model.

[0093] In some embodiments, during model training, image patches consisting of all pixels in the holed patch are used as the ground truth labels. The network loss is calculated after each training round, which is a weighted sum of adaptive multi-scale structural similarity loss and adaptive mean squared error loss. By introducing adaptive multi-scale structural similarity loss and adaptive mean squared error loss, the model's ability to perceive and reconstruct local tissue structures is enhanced. The adaptive multi-scale structural similarity loss and adaptive mean squared error loss are adaptively balanced through learnable weights, achieving an optimal trade-off between visual quality and structural consistency. Through this weighted combination, the model not only improves quantification metrics (PSNR, SSIM) but also maintains structural integrity and physical continuity of magnetic signals in the lesion region.

[0094] As an example, one sample corresponds to one hole patch. When the hole coefficient d=2 and the patch size factor h=3, the size of the hole patch is 5×5, the hole patch contains 9 pixels, and the size of the real label is 3×3.

[0095] In some embodiments, the Adaptive Multi-scale Structural Similarity Index Measure (AM-SSIM) guides the model to focus on local details and texture fidelity by comparing the structural similarity of the reconstructed image patch with the ground truth label at multiple scales in terms of brightness, contrast, and structural components. Its calculation formula is as follows: , where L AM-SSIM This represents the adaptive multi-scale structural similarity loss, where Num represents the batch size for each training round. For example, if Num=4, AM-SSIM(B n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The multi-scale structural similarity between the samples is used to reconstruct image patches, which consist of the final rendered values ​​of all pixels in the holed patch corresponding to a sample in the training set. M represents the number of scales in the multi-scale decomposition. In this embodiment, M=3. The first scale is the original scale, the second scale is the scale after one 2x downsampling, and the third scale is the scale after two 2x downsampling. M (B n,pred B n,g.t. ) represents B n,pred With B n,g.t Structural similarity between luminance components at the M-th scale, con m (B n,pred B n,g.t. ) represents B n,pred With B n,g.t Structural similarity between contrast components at the m-th scale, str m (B n,pred B n,g.t. ) represents B n,pred With B n,g.t Structural similarity between structural components at the m-th scale, bri M (B n,pred B n,g.t. ), bri M (B n,pred B n,g.t. ) and str m (B n,pred B n,g.t. All are measured by SSIM, α M β represents the learnable weight parameter of the luminance component at the M-th scale. m τ represents the learnable weight parameters of the contrast component at the m-th scale. m This represents the learnable weight parameters of the structural components at the m-th scale, with preset initial values ​​for these learnable weight parameters.

[0096] In some embodiments, the Adaptive Mean Squared Error (A-MSE) constrains global intensity consistency at the pixel level to prevent over-smoothing, and its calculation formula is as follows: , where L A-MSE A-MSE(B) represents the adaptive mean squared error loss. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The mean square error between them B represents B n,pred With B n,g.t The number of pixels in λ is used to alleviate overfitting caused by multilayer perceptrons. P is the 2-norm operator. b,g.t. B n,g.t The pixel value of the b-th pixel in the image, P b,z B n,pred The pixel value of the b-th pixel in the image, P b,s Indicates with B n,g.t Corresponding initial rendered image block B n,s,pred The pixel value of the b-th pixel in the image is used to initially render the image patch, which consists of the initial render values ​​of all pixels in the holed patch corresponding to a sample in the training set. The initial render value is generated as follows: based on N... c The first color value and first volume density of each first sampling point are used to generate N using spherical volume rendering technology. c The initial rendering value of the pixel corresponding to the first sampling point.

[0097] As an example, generate N c The formula for the initial rendering value of the pixel corresponding to the first sampling point is: , where P s ω represents the initial rendering value of the pixels corresponding to all first sampling points. i,1 This represents the first weighted value of the i-th first sampling point, and its calculation formula is given in step S221, c i,1 This represents the first color value of the i-th first sampling point.

[0098] In fact, preliminary rendering is performed based on the first sampling point, and final rendering is performed based on the second sampling point. This hierarchical spherical rendering strategy is equivalent to simulating the absorption and transmission behavior of light passing through a semi-transparent volume, thereby achieving high-fidelity rendering of three-dimensional spatial signals.

[0099] S5. The trained model is used to predict the reconstruction set. Based on the final rendered values, reconstructed image patches corresponding to the hole patches are generated, thus obtaining high-resolution SWI volumetric data with an increased number of Z-axis slices. For any sample in the reconstruction set, the trained model is input, and the final rendered values ​​of each pixel in the hole patch corresponding to that sample are output. Based on these final rendered values, a reconstructed image patch corresponding to the hole patch can be generated. Then, new slices can be generated using existing techniques. By inserting new slices in continuous coordinate space, the slice spacing along the Z-axis is reduced, achieving volumetric super-resolution reconstruction in continuous space. The inserted new slices and the original slices together form high-resolution SWI volumetric data with an increased number of Z-axis slices. High-resolution SWI volumetric data is visually characterized by natural interlayer transitions, sharp details, and very few artifacts. It maintains good morphological continuity, especially in cerebral microbleeds and vascular regions, and can be directly applied to subsequent cerebral microbleed segmentation or automatic detection tasks, significantly improving the accuracy and reliability of clinical diagnosis.

[0100] The embodiments of the present invention realize a zero-shot learning closed loop from single low-resolution SWI volume data to high-fidelity 3D reconstruction. Without the need for high-resolution paired data, it can still obtain SWI volumes with accurate structure, true signal and rich details, providing important technical support for the automatic detection and quantitative analysis of cerebral microbleeds.

[0101] To further demonstrate the feasibility and effectiveness of the method of the present invention, experiments were conducted.

[0102] This experiment compares and evaluates the method of this invention with three types of super-resolution methods. Traditional slice interpolation techniques include cubic interpolation and trilinear interpolation; super-resolution methods based on implicit neural representations, particularly SA-INR (from X. Wang et al., "Spatial attention-based implicit neural representation for arbitrary reduction of MRI slice spacing," Med. ImageAnal., vol. 94, p. 103158, 2024), which is a supervised learning method requiring paired low-resolution and high-resolution training data; and methods based on neural radiation fields, namely NeRF and CuNeRF. The neural radiation field model of this invention is implemented in PyTorch, and all experiments were performed on a single NVIDIA RTX 3090 GPU (24GB VRAM). The optimizer used was Adam, with a weight decay set to 10. -6 The maximum number of training iterations is 50,000, the batch size is 4, and training ends when the network loss converges or the maximum number of training iterations is reached. The hole patch size is 32×32, and the learning rate starts from 2×10⁻⁶. -3 It begins to decrease logarithmically to 2 × 10 -5 .

[0103] Due to computational resource considerations, the training phase of the neural radiation field model employs an 8-layer MLP network with approximately 0.5M parameters. The model first samples 64 points (i.e., N) through a coarse MLP, i.e., the first multilayer perceptron. c =64), then the ordered union of the 128 sampling points (i.e., N) f =64, total number of sampling points N c +N f=64+64=128) Input fine MLP, i.e., the second multilayer perceptron. The training time for each H×W×L=384×384×72 volume data is approximately 2.5 hours, where H represents height, W represents width, and L represents the number of slices. During the testing phase, considering computational efficiency and the model already being fitted, the neural radiation field model was set to sample 16 points, of which 8 points were used for the coarse MLP (i.e., the first multilayer perceptron) and 8 points were used for the fine MLP (i.e., the second multilayer perceptron). The inference time for rendering 384×384×144 volume data is approximately 45 seconds. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used as two quantitative metrics for performance measurement.

[0104] I. SWI Super-Resolution Results

[0105] 1. Quantitative analysis

[0106] Table 1 summarizes the quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the LHL-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4. Table 2 summarizes the quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the HKCU-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0107] Table 1. Quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the LHL-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0108]

[0109] Table 2. Quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the HKCU-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0110]

[0111] At all scales, the method of this invention consistently achieves the highest PSNR and SSIM scores, demonstrating its superior fidelity and structure preservation capabilities in reconstructing SWI volumetric data. Specifically, on the LHL-SWI-CMB dataset, the PSNR at ×2, ×3, and ×4 upsampling folds is 1.53 dB, 1.06 dB, and 0.58 dB higher than the best competing method (CuNeRF), respectively; a similar trend was observed on the HKCU-SWI-CMB dataset, proving the adaptability of the method to different acquisition devices and patient populations. Compared to other methods, NeRF exhibits a significant performance gap, primarily attributed to its ray sampling strategy failing to provide sufficient spatial coverage for medical volumetric data. The experimental results further emphasize that, unlike natural images, medical images typically contain only three orthogonal slice orientations, lacking the multi-view information crucial for effective rendering of neural radiation fields.

[0112] 2. Qualitative Analysis

[0113] Figure 3 The reconstruction results of axial, coronal, and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention are presented on the LHL-SWI-CMB dataset. Figure 4 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification are shown on the HKCU-SWI-CMB dataset using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention. Figure 3 and Figure 4 In the diagram, the columns from left to right represent the reconstruction results of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention, as well as the true labels; the rows from top to bottom represent the reconstruction results and magnification effects of axial slices, coronal slices, and sagittal slices.

[0114] from Figure 3 and Figure 4As can be seen, compared with traditional interpolation techniques, the method of this invention can restore finer anatomical structures and maintain the contrast of CMB lesions. Specifically, cubic and trilinear interpolation produce an overly smoothed appearance, blurring delicate vascular boundaries. The SA-INR method partially restores local texture. The NeRF method introduces significant mesh-like artifacts, especially in the high-frequency region, thus reducing diagnostic reliability. Although CuNeRF alleviates some distortion and presents clearer vascular contours, slight blurring remains around the lesion boundaries. In contrast, the method of this invention can reconstruct sharper edges, higher structural integrity, and more realistic SWI signal changes within the CMB region (highlighted in yellow), most closely resembling reality. Figure 4 Further detailed analysis of the axial and coronal views shows that for extremely small CMB lesions, the reconstruction results obtained by the method of this invention are remarkably similar to the actual situation in both morphology and appearance. These visual results demonstrate that the method of this invention not only improves overall clarity but also preserves microbleed features that are of significant clinical importance for neuroimaging analysis.

[0115] II. T1 Super-Resolution Results

[0116] To further verify the effectiveness of the method of the present invention, super-resolution experiments were also conducted in the T1 mode.

[0117] Figure 5 The reconstruction results of axial, coronal, and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention are presented on the MSD-T1 dataset. Figure 5 In the diagram, the columns from left to right represent the reconstruction results of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention, as well as the true labels; the rows from top to bottom represent the reconstruction results and magnification effects of axial slices, coronal slices, and sagittal slices.

[0118] Compared to SWI modality images, T1 modality images contain relatively stable tissue information, have a more uniform signal distribution, and do not depict the complex vascular networks, microbleeds, calcifications, and other subtle structural features found in SWI. Figure 5 From an overall visual perspective, all methods can preserve the basic structural features of brain tissue well in T1 modality image reconstruction. In brain tumor reconstruction tasks, the reconstruction results of the method of this invention are most consistent with the real situation, and this advantage is particularly evident in axial and coronal views.

[0119] III. Downstream Tasks

[0120] To further evaluate the effectiveness of the method of the present invention, it was applied to a downstream task of CMB lesion segmentation. The performance of this task directly reflects the quality of the reconstructed high-resolution image. Specifically, by analyzing the difference between the segmentation results obtained based on the super-resolution generated image and the real high-resolution image, it is possible to evaluate whether the method of the present invention preserves the key anatomical features and subtle details in the original data.

[0121] For the CMB lesion segmentation task, the E proposed by Xia et al. was adopted. 2 U-Net was used as the segmentation network. Experiments were conducted using the LHL-SWI-CMB dataset. U-Net was used to train E... 2 U-Net divides the LHL-SWI-CMB dataset into training and testing sets in an 8:2 ratio, that is, it uses 27 SWI volume data to train the segmentation network and retains 7 SWI volume data for testing.

[0122] The seven test cases were further processed as follows: First, they were downsampled by a factor of 2 to generate corresponding low-resolution images; then, different super-resolution methods (including cubic interpolation, trilinear interpolation, SA-INR, NeRF and the method of this invention) were used to perform super-resolution reconstruction on these low-resolution images to obtain super-resolution images, which were then divided into five super-resolution test datasets.

[0123] Apply the trained E 2 The U-Net model was split across seven test datasets: the original high-resolution test dataset, a downsampled low-resolution test dataset, and five super-resolution test datasets generated by different reconstruction methods. The effectiveness of each super-resolution method was evaluated using the results from the high-resolution and low-resolution test datasets as baselines. Figure 6 This paper presents a statistical comparison of segmentation performance across different super-resolution test datasets. Histograms and line graphs clearly illustrate the improvement in Dice scores compared to low-resolution (LR) data, as well as the remaining gap with high-resolution (HR) data. The differences in segmentation performance between images generated by different super-resolution methods highlight the crucial impact of reconstruction quality on downstream tasks.

[0124] Overall, the results of downstream tasks involving CMB lesion segmentation confirm the effectiveness of the method of this invention, which can better preserve diagnostic-related features and achieve more accurate automatic detection of cerebral microbleeds, thereby providing reliable technical support for clinical applications.

[0125] IV. Conclusion

[0126] This invention presents a zero-shot, arbitrary-scale super-resolution reconstruction method specifically designed to address key limitations of existing methods, particularly inadequate spatial modeling, poor sampling strategies for magnetically sensitive pathological structures, and insufficient preservation of fine structural details. The method introduces three core innovations to address these challenges: First, it proposes a hole patch sampling strategy to construct continuous image patches, thereby enhancing spatial context modeling and local detail reconstruction; second, it proposes a spherical sampling strategy that leverages the radial attenuation characteristics of the magnetic susceptibility signal to effectively capture the circular propagation pattern of magnetic field disturbances; third, it employs a hybrid loss method that integrates adaptive multi-scale structural similarity loss and adaptive mean square error loss to enhance structure awareness and improve reconstruction fidelity. Extensive experiments demonstrate that this method achieves world-leading performance in reconstructing high-resolution images from low-resolution magnetic resonance imaging inputs, particularly excelling in preserving key diagnostic pathological structures. This framework provides a new direction for zero-shot, physically-driven magnetic resonance super-resolution research, without requiring task-specific training.

[0127] It should be noted that, due to limitations in computing resources, the number of fine sampling points in the method of this invention is set to N. f =64, no further evaluation was conducted for higher sampling densities. However, compared to CuNeRF (N f =128), the method of this invention achieves superior reconstruction performance on all datasets with fewer sampling points. This shows that the method of this invention can not only run efficiently with limited computing resources, but also exhibits strong performance improvement potential when additional computing power is obtained.

Claims

1. A neural radiated field volume super-resolution reconstruction method for brain microhemorrhage detection, characterized in that, The method comprises the following steps: S1, acquiring low-resolution SWI volume data of a patient, performing multi-scale coordinate sampling on slices, comprising: Local sparse coordinate sampling: repeatedly sampling multiple different hollow patches on each slice, each hollow patch providing a first coordinate set containing a central pixel and its periphery sparse distributed pixels; Three-dimensional radial coordinate sampling: for each coordinate in the first coordinate set, taking multiple first sampling points in a three-dimensional spherical space with the coordinate as the center and a preset radius, and constructing a second coordinate set by Cartesian coordinates of the first sampling points; Aggregating all second coordinate sets corresponding to one hollow patch into one sample; S2, constructing a neural radiance field model, wherein a multi-layer perception is used to learn the mapping from Cartesian coordinates to color values and volume densities; Inputting each second coordinate set in the sample into the model, and combining with the spherical volume rendering technology to generate the final rendering value of the corresponding pixel, and outputting; In the step S2, the specific process of inputting each second coordinate set in the sample into the model and combining with the spherical volume rendering technology to generate the final rendering value of the corresponding pixel is: S21, inputting each second coordinate set in the sample into the first multi-layer perception in the model to obtain the first color value and the first volume density of the first sampling point; S22, based on the first volume density, sampling multiple second sampling points in the same three-dimensional spherical space; The sampling process of the second sampling point in the step S22 is: S221, calculating the first weighting value of each first sampling point based on the first volume density of all first sampling points; S222, arranging the first weighting values of all first sampling points in descending order, and taking the radial distance of the first sampling points corresponding to the first weighting values in the three-dimensional spherical space; S223, according to the two random variables introduced, which are independent of each other and subject to [0, 1] uniform distribution, respectively calculating the azimuth angle and the polar angle, and combining the radial distance obtained in the step S222 to determine a second sampling point; S224, repeating the process of step S223 to sample multiple second sampling points at different radial distances; S23, inputting the Cartesian coordinates of the two types of sampling points into the second multi-layer perception in the model to obtain the second color value and the second volume density of the corresponding sampling points, and generating the final rendering value of the pixel corresponding to all sampling points through the spherical volume rendering technology; S3, selecting one slice from every a-1 slices in the low-resolution SWI volume data according to a preset Z-axis super-division multiple a, and constructing a training set by the samples corresponding to all selected slices, and constructing a to-be-reconstructed set by the samples corresponding to the remaining slices; S4, training the model based on the training set to obtain a trained model; S5, using the trained model to predict the to-be-reconstructed set, generating a reconstructed image block corresponding to the hollow patch according to the output final rendering value, and then obtaining high-resolution SWI volume data with increased number of Z-axis slices.

2. The method of claim 1, wherein, The sampling process of the hole patch in the step S1 is: a pixel in the slice is randomly selected as a center pixel, and around the center pixel, a hole patch with a size of ((h-1)d+1)×((h-1)d+1) is sampled according to a preset patch size factor h and a preset hole coefficient d.

3. The method of claim 1, wherein, The process of sampling a first sampling point in the three-dimensional spherical space in the step S1 is: according to three random variables introduced, which are independent of each other and subject to [0, 1] uniform distribution, a radial distance, an azimuth angle and a polar angle are calculated respectively to determine a first sampling point.

4. The method of claim 3, wherein, The radial distance r = R x u 1 / 3 The azimuth angle θ = 2π x δ, and the polar angle φ = arccos(2η - 1), wherein u, δ, and η are three random variables, and R represents a preset radius.

5. The method of claim 1, wherein, The first weighting value ω i,1 = T i,1 (1-exp(-σ i,1 (r i+1 -r i ))) where i=1,2,…,N c , N c represents the number of first sampling points, ω i,1 represents the first weighting value of the i-th first sampling point, T i,1 represents the first transmittance of the i-th first sampling point, , σ i,1 represents the first volume density of the i-th first sampling point, r i represents the radial distance of the i-th first sampling point within the three-dimensional spherical space where it is located, i=1,2,…,N c -1 r i+1 represents the radial distance of the i+1-th first sampling point within the three-dimensional spherical space where it is located, i=N c -1 r i+1 =r i .

6. The neural radiated field volume super-resolution reconstruction method for brain microhemorrhage detection according to claim 5, characterized in that, The formula for generating the final rendering value of the pixel corresponding to all sampling points in step S23 through the spherical body rendering technique is: wherein P z represents the final rendering value of the pixel corresponding to all sampling points, r j represents the radial distance of the jth sampling point in the three-dimensional spherical space where it is located, ω j,2 represents the second weighting value of the jth sampling point, ω j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2 represents the second transmittance of the jth sampling point, , σ j,2 represents the second volume density of the jth sampling point, j=1,2,…,N c +N f -1 r j+1 represents the radial distance of the j+1th sampling point in the three-dimensional spherical space where it is located, j=N c +N f r j+1 =r j , σ k,2 represents the second volume density of the kth sampling point, c j,2 represents the second color value of the jth sampling point.

7. The method of claim 1, wherein, During the model training, an image block composed of all pixels in the hole patch is taken as a real label, and a network loss is calculated after each round of training, which is a weighted sum of an adaptive multi-scale structural similarity loss and an adaptive mean square error loss.

8. The method of claim 7, wherein, The calculation formula of the adaptive multi-scale structural similarity loss is as follows: , wherein L AM-SSIM denotes the adaptive multi-scale structural similarity loss, Num denotes the batch size of each round of training, AM-SSIM(B n,pred ,B n,g.t. ) denotes the multi-scale structural similarity between the nth reconstructed image block B n,pred and the corresponding real label B n,g.t. in a batch, M denotes the scale number of multi-scale decomposition, bri M (B n,pred ,B n,g.t. ), con m (B n,pred ,B n,g.t. ), str m (B n,pred ,B n,g.t. ) correspond to the structural similarity between the brightness, contrast, and structural components of B n,pred and B n,g.t. at the corresponding scale, and α M , β m , τ m correspond to the learnable weight parameters of the brightness, contrast, and structural components at the corresponding scale.

9. The neural radiated field volume super-resolution reconstruction method for brain microhemorrhage detection of claim 8, wherein, The calculation formula of the adaptive mean square error loss is: , wherein L A-MSE represents the adaptive mean square error loss, A-MSE(B n,pred , B n,g.t. ) represents the mean square error between the nth reconstructed image block B n,pred in a batch and the corresponding true label B n,g.t. , , B represents the pixel number in B n,pred and B n,g.t. , , P b,g.t. represents the pixel value of the bth pixel in B n,g.t. , P b,z represents the pixel value of the bth pixel in B n,pred , and P b,s represents the pixel value of the bth pixel in the preliminary rendered image block B n,g.t. corresponding to B n,s,pred , which is composed of the preliminary rendering values of all pixels in the hole patch corresponding to the same sample in the training set.

Images