Nerve radiation field volume super-resolution reconstruction method for brain micro-hemorrhage detection

Abstract

The invention discloses a neural radiation field volume super-resolution reconstruction method for brain micro-hemorrhage detection. The method comprises the following steps: firstly, acquiring low-resolution SWI volume data of a patient, and constructing a sample through multi-scale coordinate sampling: performing local sparse coordinate sampling on a slice to obtain a first-level coordinate set of a hole patch, and then performing three-dimensional radial coordinate sampling on each coordinate to obtain a second-level coordinate set; then constructing a neural radiation field model, inputting a sample into the model, and generating a final rendering value in combination with a spherical body rendering technology; according to a preset super-division multiple a, slices are selected at intervals to form a training set and a to-be-reconstructed set, the to-be-reconstructed set is predicted after the model is trained, and finally the high-resolution SWI volume data with the increased Z-axis slice number is obtained. According to the method, paired data training is not needed, the local structure reconstruction fidelity is remarkably improved through an innovative sampling strategy, the specific magnetic sensitive characteristic of the SWI can be accurately reconstructed, and the defects in the prior art are effectively overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing and computer vision, and particularly relates to a neural radiance field volume super-resolution reconstruction method for cerebral microbleed detection. BACKGROUND

[0002] Susceptibility weighted imaging (SWI) can sensitively detect cerebral microbleeds (CMB), but its image quality is limited by the magnetic field strength and scanning time. In order to reduce the cost or reduce the motion artifacts, a large layer thickness acquisition protocol is often used in clinic, which leads to a decrease in image resolution and a significant anisotropy, and the coronal and sagittal structures are blurred, which seriously restricts the lesion interpretation and automatic detection performance.

[0003] For the super-resolution reconstruction of low-resolution SWI, the existing technology has experienced the evolution from interpolation method to deep learning. In recent years, implicit neural representation (INR) has realized the reconstruction of any scale, but its global modeling is difficult to capture the complex local details of medical images. Although improved research has introduced convolutional neural networks to enhance details, such methods rely on a large amount of paired data for training, and the low-resolution images synthesized based on simple down-sampling cannot simulate the artifacts and noise of real clinical acquisition, resulting in insufficient generalization ability of the model in real scenarios.

[0004] As an important branch of INR, neural radiance field (NeRF) realizes the high-quality reconstruction of complex three-dimensional scenes by jointly modeling the volume density and color information of spatial coordinates, and combining position encoding and hierarchical sampling. Based on this, studies such as CuNeRF have introduced the NeRF framework into the task of medical image volume super-resolution reconstruction, realizing zero-shot, arbitrary-scale super-resolution reconstruction on a single low-resolution input, and reducing the dependence on paired data. However, existing NeRF-based methods still have obvious limitations in medical image, especially SWI volume data reconstruction: first, its supervised method based on random rays or cubic sampling ignores the spatial correlation between pixels / voxels, lacks the ability to model the overall structure of the image block, and is prone to over-smoothing and loss of local details in the reconstruction result; second, the standard NeRF sampling strategy fails to fully consider the radial signal variation characteristics caused by the difference in magnetic susceptibility in SWI images, and the reconstruction effect of the low-signal "halo" and other magnetic susceptibility artifacts around CMB is not good, and these structures are crucial for CMB diagnosis.

[0005] In summary, the existing super-resolution reconstruction methods still face three major challenges when applied to CMB detection: (1) Over-reliance on paired high-low resolution training data; (2) Insufficient local structure preservation in SWI volume data; (3) The magnetic susceptibility physical imaging characteristics specific to SWI are not fully considered, resulting in an unsatisfactory reconstruction effect of CMB related subtle structures. SUMMARY

[0006] The technical problem to be solved by the present application is to provide a neural radiance field volume super-resolution reconstruction method for brain microbleed detection, which can significantly improve the reconstruction fidelity of local structures without paired high-low resolution training data, and accurately reconstruct the magnetic susceptibility characteristics specific to SWI (susceptibility weighted imaging), thereby overcoming the shortcomings of the prior art.

[0007] The technical solution adopted by the present application to solve the above technical problem is: a neural radiance field volume super-resolution reconstruction method for brain microbleed detection, comprising the following steps: S1, obtaining low-resolution SWI volume data of a patient, and performing multi-scale coordinate sampling on the slices, including: Local sparse coordinate sampling: repeatedly sampling a plurality of different hole patches on each slice, and each hole patch provides a first coordinate set containing a central pixel and its periphery sparse distributed pixels; Three-dimensional radial coordinate sampling: for each coordinate in the first coordinate set, taking a plurality of first sampling points in a three-dimensional spherical space with the coordinate as the center and a preset radius, and constructing a second coordinate set by the Cartesian coordinates of the first sampling points; Aggregating all the second coordinate sets corresponding to one hole patch into a sample; S2, constructing a neural radiance field model, wherein a multi-layer perception is used to learn the mapping from Cartesian coordinates to color values and volume density; Inputting each second coordinate set in the sample into the model, and combining with the spherical volume rendering technology to generate the final rendering value of the corresponding pixel, and outputting; S3, selecting one slice every a-1 slices from the low-resolution SWI volume data according to a preset Z-axis super-resolution multiple a, and constructing a training set by the samples corresponding to all selected slices, and constructing a to-be-reconstructed set by the samples corresponding to the remaining all slices; S4, training the model based on the training set to obtain a trained model; S5, predicting the to-be-reconstructed set by using the trained model, generating a reconstructed image block corresponding to the hole patch according to the output final rendering value, and then obtaining high-resolution SWI volume data with the number of Z-axis slices increased.

[0008] In one possible implementation, the sampling process of the hole patch in step S1 is as follows: a pixel in the slice is randomly selected as a center pixel, and around the center pixel, a hole patch with a size of ((h-1)d+1)×((h-1)d+1) is sampled according to a preset patch size factor h and a preset hole coefficient d. The present application converts the innovative local sparse coordinate sampling idea into a clear, quantifiable, and repeatable implementation scheme by limiting the sampling of the hole patch.

[0009] In one possible implementation, the process of sampling a first sampling point in a three-dimensional spherical space in step S1 is as follows: according to three random variables introduced, which are independent of each other and subject to a uniform distribution of [0, 1], a radial distance, an azimuth angle, and a polar angle are calculated respectively to determine a first sampling point. The present application ensures that the three-dimensional radial coordinate sampling can truly reflect the distribution of the spherical symmetric physical field, and avoids the model learning deviation caused by uneven sampling, thereby laying a solid foundation for subsequent high-fidelity volume rendering.

[0010] In one possible implementation, the radial distance r=R×u 1 / 3 , the azimuth angle θ=2π×δ, and the polar angle φ=arccos(2η-1), where u, δ, and η are three random variables, and R represents a preset radius.

[0011] In one possible implementation, in step S2, the specific process of inputting each secondary coordinate set in the sample into the model and generating the final rendering value of the corresponding pixel in combination with the spherical volume rendering technology is as follows: S21, inputting each secondary coordinate set in the sample into a first multi-layer perception machine in the model to obtain a first color value and a first volume density of the first sampling point; S22, sampling a plurality of second sampling points in the same three-dimensional spherical space based on the first volume density; S23, inputting the Cartesian coordinates of the two types of sampling points into a second multi-layer perception machine in the model to obtain a second color value and a second volume density of the corresponding sampling point, and generating the final rendering value of all sampling points corresponding to the pixel through the spherical volume rendering technology.

[0012] In one possible implementation, the sampling process of the second sampling point in step S22 is as follows: S221, calculating a first weighting value ω i,1 i i,1 of each first sampling point based on the first volume density of all first sampling points, where i=1, 2, …, N i,1 . i+1 i ​​c , N c represents the number of first sampling points, ω i,1 represents the first weighting value of the i-th first sampling point, T i,1 represents the first transmittance of the i-th first sampling point, , σ i,1 represents the first bulk density of the i-th first sampling point, r i represents the radial distance of the i-th first sampling point in the three-dimensional spherical space where it is located, i = 1, 2, …, N c -1, r i+1 represents the radial distance of the i+1-th first sampling point in the three-dimensional spherical space where it is located, i = N c +N i+1 -1, r i ; S222, arrange the first weighting values of all first sampling points in descending order, and take the radial distances of the first sampling points corresponding to the first weighting values in the front in the three-dimensional spherical space where they are located; S223, according to the two random variables introduced, which are independent of each other and subject to [0, 1] uniform distribution, respectively calculate the azimuth angle and the polar angle, and combine the radial distance obtained in step S222 to determine a second sampling point; S224, repeat the process of step S223 to sample multiple second sampling points at different radial distances.

[0013] In one possible implementation, the formula for generating the final rendering value of the pixel corresponding to all sampling points through the spherical volume rendering technique in step S23 is: , wherein P z represents the final rendering value of the pixel corresponding to all sampling points, r j represents the radial distance of the j-th sampling point in the three-dimensional spherical space where it is located, ω j,2 represents the second weighting value of the j-th sampling point, ω j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2 represents the second transmittance of the j-th sampling point, , σ j,2 represents the second bulk density of the j-th sampling point, j = 1, 2, …, N c +N f -1, r j+1 represents the radial distance of the j+1-th sampling point in the three-dimensional spherical space where it is located, j = N c +N f -1, r j+1 =rj , σ k,2 denotes the second body density of the k-th sampling point, c j,2 denotes the second color value of the j-th sampling point.

[0014] In one possible implementation, the model is trained with image blocks composed of all pixels in the hole patch as real labels, and the network loss is calculated after each round of training, which is the weighted sum of adaptive multi-scale structural similarity loss and adaptive mean square error loss.

[0015] In one possible implementation, the calculation formula of the adaptive multi-scale structural similarity loss is: , wherein L AM-SSIM denotes the adaptive multi-scale structural similarity loss, Num denotes the batch size of each round of training, AM-SSIM(B n,pred ,B n,g.t. ) denotes the multi-scale structural similarity between the n-th reconstructed image block B n,pred and the corresponding real label B n,g.t in a batch, , M denotes the number of scales of multi-scale decomposition, bri M (B n,pred ,B n,g.t. ), con m (B n,pred ,B n,g.t. ), str m (B n,pred ,B n,g.t. ) correspond to the structural similarity between the luminance, contrast, and structural components of B n,pred and B n,g.t at the corresponding scale, α M , β m , τ m correspond to the learnable weight parameters of the luminance, contrast, and structural components at the corresponding scale.

[0016] In one possible implementation, the calculation formula of the adaptive mean square error loss is: , wherein L A-MSE denotes the adaptive mean square error loss, A-MSE(B n,pred ,B n,g.t. ) denotes the mean square error between the n-th reconstructed image block B n,pred and the corresponding real label B n,g.t in a batch, , B denotes the number of pixels in B n,pred and B n,g.t , , P b,g.t. denotes Bn,g.t pixel value of the bth pixel in B b,z denotes B n,pred pixel value of the bth pixel in B b,s denotes B n,g.t corresponding preliminary rendered image block B n,s,pred pixel value of the bth pixel in B, which is composed of the preliminary rendered values of all pixels in the hole patch corresponding to the sample in the training set.

[0017] In model training, the application introduces a composite loss function combining adaptive multi-scale structural similarity loss and adaptive mean square error loss, effectively alleviates overfitting through common supervision of preliminary rendering and final rendering, ensures robust optimization of the model, and enhances the model's perception and reconstruction ability of local organizational structure.

[0018] Compared with the prior art, the application has the following advantages: The application realizes zero-shot, arbitrary scale super-resolution reconstruction without paired data. The application selects one slice every a-1 slices from low-resolution SWI body data according to a preset Z-axis super-resolution multiple a to construct a training set and a set to be reconstructed, and realizes self-supervised learning by using the spatial continuity of the data itself. The application does not require any external paired high-low resolution training data, fundamentally solves the problem of excessive dependence on paired data in existing methods, and significantly improves the reconstruction generalization ability for real low-resolution clinical data because training and prediction are performed in the same data domain.

[0019] The application significantly improves the fidelity of local structure by multi-scale coordinate sampling. The local sparse coordinate sampling strategy adopted by the application enables the model to perceive the context information on the local neighborhood by obtaining a first coordinate set containing the center pixel and its peripheral sparse distributed pixels, thereby improving the reconstruction accuracy of local fine structure. This discards the isolated pixel sampling method in traditional NeRF, so that the final rendered value reconstructed can accurately retain the morphology and structure of subtle lesions such as cerebral microbleeding, effectively avoiding over-smoothing, and generating reconstructed image blocks and high-resolution SWI body data with more diagnostic value.

[0020] The application accurately reconstructs the magnetic susceptibility characteristics through a physically driven sampling and rendering mechanism. The three-dimensional radial coordinate sampling and spherical body rendering technology introduced by the application samples multiple first sampling points in the three-dimensional spherical space of each coordinate and performs rendering, which adapts the general neural radiance field from rendering visual appearance to reconstructing physical field signals. This enables the neural radiance field model constructed to specifically learn and accurately generate signal radial distribution caused by magnetic susceptibility difference, so as to clearly present features such as the low signal "halo" around the cerebral microbleeding lesion in the final reconstruction result.

[0021] Through the aforementioned synergistic effect, high-resolution SWI volume data with an increased number of Z-axis slices was finally reconstructed, providing a reliable imaging tool for the accurate detection of cerebral microbleeds.

[0022] This invention achieves flexible, arbitrary-scale volumetric super-resolution by presetting a super-resolution factor 'a' to increase the resolution by any factor in the Z-axis direction.

[0023] This invention is applicable to the reconstruction of microscopic magnetically sensitive structures, such as cerebral microbleeds (CMBs) and vascular networks, thereby achieving high-quality super-resolution reconstruction of low-resolution SWI volume data. Extensive experiments on SWI datasets targeting cerebral microbleeds demonstrate that this invention consistently outperforms existing methods in arbitrary-scale super-resolution reconstruction tasks, particularly excelling in preserving tissue texture and structure, while exhibiting good robustness and high accuracy in CMB lesion reconstruction. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a block diagram illustrating the implementation of the method of the present invention; Figure 2 A schematic diagram of a hole patch and multiple first sampling points on a slice; Figure 3 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the LHL-SWI-CMB dataset; Figure 4 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the HKCU-SWI-CMB dataset; Figure 5 The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention on the MSD-T1 dataset; Figure 6 A statistical comparison diagram of segmentation performance for different super-resolution test datasets. Detailed Implementation

[0026] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0027] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0028] Existing super-resolution reconstruction methods have the following technical problems when applied to CMB detection: over-reliance on paired high- and low-resolution training data; insufficient ability to preserve local structures in SWI volume data; and failure to fully consider the unique magnetic susceptibility physical imaging characteristics of SWI, resulting in unsatisfactory reconstruction results of CMB-related fine structures.

[0029] To address the problems of existing technologies, this invention provides a method for super-resolution reconstruction of neural radiation field volume for detecting brain microbleeds. This method can significantly improve the reconstruction fidelity of local structures and accurately reconstruct the magnetic susceptibility characteristics unique to SWI without requiring paired high- and low-resolution training data.

[0030] The following is a detailed description of a neural radiation field volume super-resolution reconstruction method for detecting cerebral microbleeds provided by an embodiment of the present invention.

[0031] like Figure 1 As shown, the method provided in this embodiment of the invention includes the following steps: S1. Obtain low-resolution SWI volume data from a patient, and perform multi-scale coordinate sampling on the slices contained therein to construct a sample. The multi-scale coordinate sampling includes: Local sparse coordinate sampling: Multiple different hole patches are repeatedly sampled on each slice, and each hole patch provides a first-level coordinate set containing the center pixel and its sparsely distributed surrounding pixels.

[0032] Three-dimensional radial coordinate sampling: For each coordinate in each primary coordinate set, multiple first sampling points are sampled in a three-dimensional spherical space with a preset radius and centered on it, and their Cartesian coordinates are used to form a secondary coordinate set.

[0033] All secondary coordinates corresponding to a hole patch are aggregated into a single sample.

[0034] Figure 2 This shows a hole patch on the slice and multiple first sampling points.

[0035] Here, low-resolution SWI volume data refers to SWI volume data with a small number of slices in the Z-axis direction.

[0036] In some embodiments, low-resolution SWI volume data are acquired from clinical magnetic resonance imaging equipment, which typically has a thicker slice thickness, i.e., a lower inter-slice resolution.

[0037] In some embodiments, after obtaining low-resolution SWI volume data, de-identification and normalization processes are performed first to ensure data privacy and strength consistency.

[0038] In some embodiments, the process of sampling a hole patch on a slice is as follows: randomly select a pixel in the slice as the center pixel, and around the center pixel, according to the preset patch size factor h and the preset hole coefficient d, sample a hole patch of size ((h-1)d+1)×((h-1)d+1).

[0039] In some embodiments, the sampling of hole patches is random, repeated, and ensures that all sampled hole patches cover as many pixels as possible within the slice. This hole patch sampling strategy replaces the isolated and independent pixel sampling in NeRF and CuNeRF.

[0040] Here, both the patch size factor h and the hole coefficient d are preset values. The hole coefficient d is used to control the spacing between adjacent pixels. As an example, we take d=2 and h=3. In this case, there is a hole between adjacent pixels, the size of the sampled hole patch is 5×5, and the first-level coordinate set contains the coordinates of the center pixel of the hole patch and the coordinates of the 8 sparsely distributed pixels around it, that is, it contains the coordinates of a total of 9 pixels.

[0041] As an example, when a hole patch contains 9 pixels, the corresponding first-level coordinate set contains the coordinates of these 9 pixels. Taking the coordinates of the 5th pixel as an example, N is sampled within a three-dimensional spherical space of a preset radius with the 5th pixel as the center. c There are 9 first sampling points, and the Cartesian coordinates of these first sampling points are used to form a second-level coordinate set. These 9 second-level coordinate sets are then aggregated into a single sample.

[0042] In some embodiments, the process of sampling a first sampling point in a three-dimensional spherical space is as follows: three independent random variables are introduced, each of which follows a uniform distribution of [0,1]; then the radial distance, azimuth angle and polar angle in spherical coordinates are calculated based on the three random variables respectively, thereby determining a first sampling point.

[0043] As an example, if N samples are taken in a three-dimensional spherical space c If there is a first sampling point, then repeat the sampling process for the first sampling point N. c Next, due to the introduction of random variables, therefore N c Each of the first sampling points is different. To meet the sampling density requirements, in this embodiment, N samples are taken within a three-dimensional spherical space. c =64 first sampling points. This sampling strategy within a three-dimensional spherical space is consistent with the radial distribution characteristics of the magnetic susceptibility signal in SWI. Sampling on a concentric sphere achieves comprehensive spatial coverage of the surrounding radial region, effectively capturing circular or annular structural patterns, such as the low-signal halos commonly found in CMB.

[0044] In some embodiments, the radial distance r = R × u 1 / 3 The azimuth angle θ = 2π × δ, and the polar angle φ = arccos(2η-1), where u, δ, and η are three random variables, and R represents the preset radius. Here, r is used to control the distance from the sampling point to the center of the sphere, r ∈ [0, R]; θ is used to specify the horizontal direction of the sampling point, θ ∈ [0, 2π]; φ is used to specify the vertical direction of the sampling point, φ ∈ [0, π]; R is a hyperparameter, which is set to R = 1 / 2 in this embodiment.

[0045] In some embodiments, the method further includes converting the spherical coordinates of the first sampling point to Cartesian coordinates, while converting the spherical coordinates of a point to Cartesian coordinates is a conventional technique.

[0046] S2. Construct a neural radiation field model, the core of which is a multilayer perceptron, to learn the mapping from Cartesian coordinates to color values ​​and volume density.

[0047] Each set of secondary coordinates in the sample is input into the model, and combined with spherical rendering technology, the final rendered value of the corresponding pixel is generated and output.

[0048] In some embodiments, the neural radiation field model includes a first multilayer perceptron and a second multilayer perceptron. Here, the number of layers in the multilayer perceptron can be designed to be 6-9, and in a specific implementation, the number of layers is designed to be 8. The specific process of inputting each set of secondary coordinates in the sample into the model and combining it with spherical rendering technology to generate the final rendered value of the corresponding pixel is as follows: S21. Input each set of secondary coordinates in the sample into the first multilayer perceptron in the model to obtain the first color value and the first volume density of the first sampling point.

[0049] As an example, a set of two-dimensional coordinates contains N c N Cartesian coordinates, the first multilayer perceptron outputs N. c The first color value and the first volume density of the first sampling point.

[0050] S22. Based on the first volume density, continue to sample multiple second sampling points within the same three-dimensional spherical space.

[0051] In some embodiments, the sampling process for the second sampling point is as follows: S221. Based on the first volume density of all first sampling points, calculate the first weighted value of each of the first sampling points. The calculation formula is: ω i,1 =T i,1 (1-exp(-σ i,1 (r i+1 -r i ))), where i=1,2,…,N c N c ω represents the number of the first sampling points. i,1 T represents the first weighted value of the i-th first sampling point. i,1 This represents the first transmittance of the i-th first sampling point. exp represents an exponential function with base e, σ i,1 Let r represent the first volume density of the i-th first sampling point. i This represents the radial distance of the i-th first sampling point within its three-dimensional spherical space, where i = 1, 2, ..., N. c -1 hour r i+1 This represents the radial distance of the (i+1)th first sampling point within its three-dimensional spherical space, where i=N c time r i+1 =r i , here r k σ is the radial distance of the k-th first sampling point within its three-dimensional spherical space. k,1 Let r represent the first volume density of the k-th first sampling point. k This represents the radial distance of the k-th first sampling point within its three-dimensional spherical space, k=1,2,…,N c-1 hour r k+1 This represents the radial distance of the (k+1)th first sampling point within its three-dimensional spherical space, where k=N. c time r k+1 =r k .

[0052] S222. Arrange the first weighted values ​​of all first sampling points in descending order, and take the radial distance of the first sampling points corresponding to the first few first weighted values ​​in their respective three-dimensional spherical spaces.

[0053] As an example, for N c The first weighted values ​​of the first sampling points are arranged in descending order, and the first 3 first weighted values ​​are taken. It is assumed that the radial distances of the first sampling points corresponding to these first weighted values ​​in their respective three-dimensional spherical spaces are r1, r2, and r3.

[0054] S223. Introduce two independent random variables that both follow a uniform distribution in [0,1]. Then, calculate the azimuth and polar angle in spherical coordinates based on the two random variables, and determine a second sampling point by combining the radial distance obtained in step S222.

[0055] S224. Repeat step S223 to sample multiple second sampling points at different radial distances.

[0056] As an example, a second sampling point is obtained by taking a radial distance r1 and combining it with the azimuth and polar angles calculated from two random variables. At this radial distance r1, multiple random variables are taken to obtain multiple second sampling points with the same radial distance but different azimuth and polar angles. Similarly, multiple second sampling points with the same radial distance but different azimuth and polar angles are obtained at radial distances r2 and r3, respectively.

[0057] As an example, based on N c =The first volume density of the first sampling points is obtained by sampling N. f =64 second sampling points.

[0058] S23. Input the Cartesian coordinates of the two types of sampling points into the second multilayer perceptron in the model to obtain the second color value and second volume density of the corresponding sampling points. Then, through spherical volume rendering technology, generate the final rendering value of the pixels corresponding to all sampling points and output it.

[0059] In some embodiments, the formula for generating the final rendered value of the pixel corresponding to all sampling points using spherical volume rendering technology is as follows: , where P z r represents the final rendered value of the pixel corresponding to all sampling points. jω represents the radial distance of the j-th sampling point (which may be the first or second sampling point) within its three-dimensional spherical space. j,2 ω represents the second weighted value of the j-th sampling point. j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2 This represents the second transmittance at the j-th sampling point. , σ j,2 Let j represent the second volume density at the j-th sampling point, j=1,2,…,N c +N f -1 hour r j+1 This represents the radial distance of the (j+1)th sampling point within its three-dimensional spherical space, where j=N c +N f time r j+1 =r j , σ k,2 c represents the second volume density at the k-th sampling point. j,2 This represents the second color value of the j-th sampling point.

[0060] As an example, there are N sampling points of two types. c =64 first sampling points and N f =64 second sampling points, and through spherical volume rendering technology, the final rendering value of one pixel corresponding to these 128 sampling points is generated.

[0061] S3. According to the preset Z-axis super-resolution factor a, select one slice from the low-resolution SWI volume data at intervals of a-1. The samples corresponding to all selected slices constitute the training set, and the samples corresponding to all remaining slices constitute the set to be reconstructed.

[0062] In this embodiment, a = 2 or 3.

[0063] S4. Train the model based on the training set to obtain a trained model.

[0064] In some embodiments, during model training, image patches consisting of all pixels in the holed patch are used as the ground truth labels. The network loss is calculated after each training round, which is a weighted sum of adaptive multi-scale structural similarity loss and adaptive mean squared error loss. By introducing adaptive multi-scale structural similarity loss and adaptive mean squared error loss, the model's ability to perceive and reconstruct local tissue structures is enhanced. The adaptive multi-scale structural similarity loss and adaptive mean squared error loss are adaptively balanced through learnable weights, achieving an optimal trade-off between visual quality and structural consistency. Through this weighted combination, the model not only improves quantification metrics (PSNR, SSIM) but also maintains structural integrity and physical continuity of magnetic signals in the lesion region.

[0065] As an example, one sample corresponds to one hole patch. When the hole coefficient d=2 and the patch size factor h=3, the size of the hole patch is 5×5, the hole patch contains 9 pixels, and the size of the real label is 3×3.

[0066] In some embodiments, the Adaptive Multi-scale Structural Similarity Index Measure (AM-SSIM) guides the model to focus on local details and texture fidelity by comparing the structural similarity of the reconstructed image patch with the ground truth label at multiple scales in terms of brightness, contrast, and structural components. Its calculation formula is as follows: , where L AM-SSIM This represents the adaptive multi-scale structural similarity loss, where Num represents the batch size for each training round. For example, if Num=4, AM-SSIM(B n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The multi-scale structural similarity between the samples is used to reconstruct image patches, which consist of the final rendered values ​​of all pixels in the holed patch corresponding to a sample in the training set. M represents the number of scales in the multi-scale decomposition. In this embodiment, M=3. The first scale is the original scale, the second scale is the scale after one 2x downsampling, and the third scale is the scale after two 2x downsampling. M (B n,pred B n,g.t. ) represents B n,pred With B n,g.t Structural similarity between luminance components at the M-th scale, con m (B n,pred B n,g.t. ) represents B n,pred With B n,g.tStructural similarity between contrast components at the m-th scale, str m (B n,pred B n,g.t. ) represents B n,pred With B n,g.t Structural similarity between structural components at the m-th scale, bri M (B n,pred B n,g.t. ), bri M (B n,pred B n,g.t. ) and str m (B n,pred B n,g.t. All are measured by SSIM, α M β represents the learnable weight parameter of the luminance component at the M-th scale. m τ represents the learnable weight parameters of the contrast component at the m-th scale. m This represents the learnable weight parameters of the structural components at the m-th scale, with preset initial values ​​for these learnable weight parameters.

[0067] In some embodiments, the Adaptive Mean Squared Error (A-MSE) constrains global intensity consistency at the pixel level to prevent over-smoothing, and its calculation formula is as follows: , where L A-MSE A-MSE(B) represents the adaptive mean squared error loss. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The mean square error between them B represents B n,pred With B n,g.t The number of pixels in λ is used to alleviate overfitting caused by multilayer perceptrons. P is the 2-norm operator. b,g.t. B n,g.t The pixel value of the b-th pixel in the image, P b,z B n,pred The pixel value of the b-th pixel in the image, P b,s Indicates with B n,g.t Corresponding initial rendered image block B n,s,pred The pixel value of the b-th pixel in the image is used to initially render the image patch, which consists of the initial render values ​​of all pixels in the holed patch corresponding to a sample in the training set. The initial render value is generated as follows: based on N... c The first color value and first volume density of each first sampling point are used to generate N using spherical volume rendering technology.c The initial rendering value of the pixel corresponding to the first sampling point.

[0068] As an example, generate N c The formula for the initial rendering value of the pixel corresponding to the first sampling point is: , where P s ω represents the initial rendering value of the pixels corresponding to all first sampling points. i,1 This represents the first weighted value of the i-th first sampling point, and its calculation formula is given in step S221, c i,1 This represents the first color value of the i-th first sampling point.

[0069] In fact, preliminary rendering is performed based on the first sampling point, and final rendering is performed based on the second sampling point. This hierarchical spherical rendering strategy is equivalent to simulating the absorption and transmission behavior of light passing through a semi-transparent volume, thereby achieving high-fidelity rendering of three-dimensional spatial signals.

[0070] S5. The trained model is used to predict the reconstruction set. Based on the final rendered values, reconstructed image patches corresponding to the hole patches are generated, thus obtaining high-resolution SWI volumetric data with an increased number of Z-axis slices. For any sample in the reconstruction set, the trained model is input, and the final rendered values ​​of each pixel in the hole patch corresponding to that sample are output. Based on these final rendered values, a reconstructed image patch corresponding to the hole patch can be generated. Then, new slices can be generated using existing techniques. By inserting new slices in continuous coordinate space, the slice spacing along the Z-axis is reduced, achieving volumetric super-resolution reconstruction in continuous space. The inserted new slices and the original slices together form high-resolution SWI volumetric data with an increased number of Z-axis slices. High-resolution SWI volumetric data is visually characterized by natural interlayer transitions, sharp details, and very few artifacts. It maintains good morphological continuity, especially in cerebral microbleeds and vascular regions, and can be directly applied to subsequent cerebral microbleed segmentation or automatic detection tasks, significantly improving the accuracy and reliability of clinical diagnosis.

[0071] The embodiments of the present invention realize a zero-shot learning closed loop from single low-resolution SWI volume data to high-fidelity 3D reconstruction. Without the need for high-resolution paired data, it can still obtain SWI volumes with accurate structure, true signal and rich details, providing important technical support for the automatic detection and quantitative analysis of cerebral microbleeds.

[0072] To further demonstrate the feasibility and effectiveness of the method of the present invention, experiments were conducted.

[0073] This experiment compares and evaluates the method of this invention with three types of super-resolution methods. Traditional slice interpolation techniques include cubic interpolation and trilinear interpolation; super-resolution methods based on implicit neural representations, particularly SA-INR (from X. Wang et al., "Spatial attention-based implicit neural representation for arbitrary reduction of MRI slice spacing," Med. ImageAnal., vol. 94, p. 103158, 2024), which is a supervised learning method requiring paired low-resolution and high-resolution training data; and methods based on neural radiation fields, namely NeRF and CuNeRF. The neural radiation field model of this invention is implemented in PyTorch, and all experiments were performed on a single NVIDIA RTX 3090 GPU (24GB VRAM). The optimizer used was Adam, with a weight decay set to 10. -6 The maximum number of training iterations is 50,000, the batch size is 4, and training ends when the network loss converges or the maximum number of training iterations is reached. The hole patch size is 32×32, and the learning rate starts from 2×10⁻⁶. -3 It begins to decrease logarithmically to 2 × 10 -5 .

[0074] Due to computational resource considerations, the training phase of the neural radiation field model employs an 8-layer MLP network with approximately 0.5M parameters. The model first samples 64 points (i.e., N) through a coarse MLP, i.e., the first multilayer perceptron. c =64), then the ordered union of the 128 sampling points (i.e., N) f =64, total number of sampling points N c +N f =64+64=128) Input fine MLP, i.e., the second multilayer perceptron. The training time for each H×W×L=384×384×72 volume data is approximately 2.5 hours, where H represents height, W represents width, and L represents the number of slices. During the testing phase, considering computational efficiency and the model already being fitted, the neural radiation field model was set to sample 16 points, of which 8 points were used for the coarse MLP (i.e., the first multilayer perceptron) and 8 points were used for the fine MLP (i.e., the second multilayer perceptron). The inference time for rendering 384×384×144 volume data is approximately 45 seconds. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used as two quantitative metrics for performance measurement.

[0075] I. SWI Super-Resolution Results 1. Quantitative analysis Table 1 summarizes the quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the LHL-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4. Table 2 summarizes the quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the HKCU-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0076] Table 1. Quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the LHL-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0077] Table 2. Quantization performance of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention on the HKCU-SWI-CMB dataset at upsampling rates of ×2, ×3, and ×4.

[0078] At all scales, the method of this invention consistently achieves the highest PSNR and SSIM scores, demonstrating its superior fidelity and structure preservation capabilities in reconstructing SWI volumetric data. Specifically, on the LHL-SWI-CMB dataset, the PSNR at ×2, ×3, and ×4 upsampling folds is 1.53 dB, 1.06 dB, and 0.58 dB higher than the best competing method (CuNeRF), respectively; a similar trend was observed on the HKCU-SWI-CMB dataset, proving the adaptability of the method to different acquisition devices and patient populations. Compared to other methods, NeRF exhibits a significant performance gap, primarily attributed to its ray sampling strategy failing to provide sufficient spatial coverage for medical volumetric data. The experimental results further emphasize that, unlike natural images, medical images typically contain only three orthogonal slice orientations, lacking the multi-view information crucial for effective rendering of neural radiation fields.

[0079] 2. Qualitative Analysis Figure 3 The reconstruction results of axial, coronal, and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention are presented on the LHL-SWI-CMB dataset. Figure 4The reconstruction results of axial, coronal and sagittal slices at a ×2 upsampling magnification are shown on the HKCU-SWI-CMB dataset using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF and the method of this invention. Figure 3 and Figure 4 In the diagram, the columns from left to right represent the reconstruction results of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention, as well as the true labels; the rows from top to bottom represent the reconstruction results and magnification effects of axial slices, coronal slices, and sagittal slices.

[0080] from Figure 3 and Figure 4 As can be seen, compared with traditional interpolation techniques, the method of this invention can restore finer anatomical structures and maintain the contrast of CMB lesions. Specifically, cubic and trilinear interpolation produce an overly smoothed appearance, blurring delicate vascular boundaries. The SA-INR method partially restores local texture. The NeRF method introduces significant mesh-like artifacts, especially in the high-frequency region, thus reducing diagnostic reliability. Although CuNeRF alleviates some distortion and presents clearer vascular contours, slight blurring remains around the lesion boundaries. In contrast, the method of this invention can reconstruct sharper edges, higher structural integrity, and more realistic SWI signal changes within the CMB region (highlighted in yellow), most closely resembling reality. Figure 4 Further detailed analysis of the axial and coronal views shows that for extremely small CMB lesions, the reconstruction results obtained by the method of this invention are remarkably similar to the actual situation in both morphology and appearance. These visual results demonstrate that the method of this invention not only improves overall clarity but also preserves microbleed features that are of significant clinical importance for neuroimaging analysis.

[0081] II. T1 Super-Resolution Results To further verify the effectiveness of the method of the present invention, super-resolution experiments were also conducted in the T1 mode.

[0082] Figure 5 The reconstruction results of axial, coronal, and sagittal slices at a ×2 upsampling magnification using cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention are presented on the MSD-T1 dataset. Figure 5In the diagram, the columns from left to right represent the reconstruction results of cubic interpolation, trilinear interpolation, SA-INR, NeRF, CuNeRF, and the method of this invention, as well as the true labels; the rows from top to bottom represent the reconstruction results and magnification effects of axial slices, coronal slices, and sagittal slices.

[0083] Compared to SWI modality images, T1 modality images contain relatively stable tissue information, have a more uniform signal distribution, and do not depict the complex vascular networks, microbleeds, calcifications, and other subtle structural features found in SWI. Figure 5 From an overall visual perspective, all methods can preserve the basic structural features of brain tissue well in T1 modality image reconstruction. In brain tumor reconstruction tasks, the reconstruction results of the method of this invention are most consistent with the real situation, and this advantage is particularly evident in axial and coronal views.

[0084] III. Downstream Tasks To further evaluate the effectiveness of the method of the present invention, it was applied to a downstream task of CMB lesion segmentation. The performance of this task directly reflects the quality of the reconstructed high-resolution image. Specifically, by analyzing the difference between the segmentation results obtained based on the super-resolution generated image and the real high-resolution image, it is possible to evaluate whether the method of the present invention preserves the key anatomical features and subtle details in the original data.

[0085] For the CMB lesion segmentation task, the E proposed by Xia et al. was adopted. 2 U-Net was used as the segmentation network. Experiments were conducted using the LHL-SWI-CMB dataset. U-Net was used to train E... 2 U-Net divides the LHL-SWI-CMB dataset into training and testing sets in an 8:2 ratio, that is, it uses 27 SWI volume data to train the segmentation network and retains 7 SWI volume data for testing.

[0086] The seven test cases were further processed as follows: First, they were downsampled by a factor of 2 to generate corresponding low-resolution images; then, different super-resolution methods (including cubic interpolation, trilinear interpolation, SA-INR, NeRF and the method of this invention) were used to perform super-resolution reconstruction on these low-resolution images to obtain super-resolution images, which were then divided into five super-resolution test datasets.

[0087] Apply the trained E 2 The U-Net model was split across seven test datasets: the original high-resolution test dataset, a downsampled low-resolution test dataset, and five super-resolution test datasets generated by different reconstruction methods. The effectiveness of each super-resolution method was evaluated using the results from the high-resolution and low-resolution test datasets as baselines.Figure 6 This paper presents a statistical comparison of segmentation performance across different super-resolution test datasets. Histograms and line graphs clearly illustrate the improvement in Dice scores compared to low-resolution (LR) data, as well as the remaining gap with high-resolution (HR) data. The differences in segmentation performance between images generated by different super-resolution methods highlight the crucial impact of reconstruction quality on downstream tasks.

[0088] Overall, the results of downstream tasks involving CMB lesion segmentation confirm the effectiveness of the method of this invention, which can better preserve diagnostic-related features and achieve more accurate automatic detection of cerebral microbleeds, thereby providing reliable technical support for clinical applications.

[0089] IV. Conclusion This invention presents a zero-shot, arbitrary-scale super-resolution reconstruction method specifically designed to address key limitations of existing methods, particularly inadequate spatial modeling, poor sampling strategies for magnetically sensitive pathological structures, and insufficient preservation of fine structural details. The method introduces three core innovations to address these challenges: First, it proposes a hole patch sampling strategy to construct continuous image patches, thereby enhancing spatial context modeling and local detail reconstruction; second, it proposes a spherical sampling strategy that leverages the radial attenuation characteristics of the magnetic susceptibility signal to effectively capture the circular propagation pattern of magnetic field disturbances; third, it employs a hybrid loss method that integrates adaptive multi-scale structural similarity loss and adaptive mean square error loss to enhance structure awareness and improve reconstruction fidelity. Extensive experiments demonstrate that this method achieves world-leading performance in reconstructing high-resolution images from low-resolution magnetic resonance imaging inputs, particularly excelling in preserving key diagnostic pathological structures. This framework provides a new direction for zero-shot, physically-driven magnetic resonance super-resolution research, without requiring task-specific training.

[0090] It should be noted that, due to limitations in computing resources, the number of fine sampling points in the method of this invention is set to N. f =64, no further evaluation was conducted for higher sampling densities. However, compared to CuNeRF (N f =128), the method of this invention achieves superior reconstruction performance on all datasets with fewer sampling points. This shows that the method of this invention can not only run efficiently with limited computing resources, but also exhibits strong performance improvement potential when additional computing power is obtained.

Claims

1. A method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds, characterized in that, Includes the following steps: S1. Obtain low-resolution SWI volume data from a patient, and perform multi-scale coordinate sampling on the slices, including: Local sparse coordinate sampling: Multiple different hole patches are repeatedly sampled on each slice, and each hole patch provides a first-level coordinate set containing the center pixel and its sparsely distributed surrounding pixels; Three-dimensional radial coordinate sampling: For each coordinate in each primary coordinate set, multiple first sampling points are sampled in a three-dimensional spherical space with a preset radius and centered on it, and their Cartesian coordinates are used to form a secondary coordinate set; The set of all secondary coordinates corresponding to a single hole patch is aggregated into a single sample; S2. Construct a neural radiation field model, in which a multilayer perceptron is used to learn the mapping from Cartesian coordinates to color values ​​and volume density; Each set of secondary coordinates in the sample is input into the model, and combined with spherical rendering technology, the final rendering value of the corresponding pixel is generated and output. S3. According to the preset Z-axis super-resolution factor a, select one slice from the low-resolution SWI volume data every a-1 frames. The samples corresponding to all selected slices constitute the training set, and the samples corresponding to all remaining slices constitute the set to be reconstructed. S4. Train the model based on the training set to obtain a trained model; S5. The trained model is used to predict the set to be reconstructed. Based on the final output rendering value, the reconstructed image block corresponding to the hole patch is generated, thereby obtaining high-resolution SWI volume data with an increased number of Z-axis slices.

2. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 1, characterized in that, The sampling process for the hole patch in step S1 is as follows: randomly select a pixel in the slice as the center pixel, and around the center pixel, according to the preset patch size factor h and the preset hole coefficient d, sample a hole patch with a size of ((h-1)d+1)×((h-1)d+1).

3. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 1, characterized in that, The process of sampling a first sampling point in the three-dimensional spherical space in step S1 is as follows: Based on the three independent random variables that all follow a uniform distribution of [0,1], calculate the radial distance, azimuth angle and polar angle respectively, and thus determine a first sampling point.

4. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 3, characterized in that, The radial distance r = R × u 1 / 3 The azimuth angle θ = 2π × δ, the polar angle φ = arccos(2η-1), where u, δ, and η are three random variables, and R represents the preset radius.

5. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 1, characterized in that, In step S2, the specific process of inputting each set of secondary coordinates from the sample into the model and generating the final rendered value of the corresponding pixel using spherical rendering technology is as follows: S21. Input each set of secondary coordinates in the sample into the first multilayer perceptron in the model to obtain the first color value and the first volume density of the first sampling point; S22. Based on the first volume density, sample multiple second sampling points within the same three-dimensional spherical space; S23. Input the Cartesian coordinates of the two types of sampling points into the second multilayer perceptron in the model to obtain the second color value and second volume density of the corresponding sampling points, and generate the final rendering value of the corresponding pixels of all sampling points through spherical volume rendering technology.

6. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 5, characterized in that, The sampling process for the second sampling point in step S22 is as follows: S221. Based on the first volume density of all first sampling points, calculate the first weighted value ω for each of the first sampling points. i,1 =T i,1 (1-exp(-σ i,1 (r i+1 -r i ))), where i=1,2,…,N c N c ω represents the number of the first sampling points. i,1 T represents the first weighted value of the i-th first sampling point. i,1 This represents the first transmittance of the i-th first sampling point. , σ i,1 Let r represent the first volume density of the i-th first sampling point. i This represents the radial distance of the i-th first sampling point within its three-dimensional spherical space, where i = 1, 2, ..., N. c -1 hour r i+1 This represents the radial distance of the (i+1)th first sampling point within its three-dimensional spherical space, where i=N c time r i+1 =r i ; S222. Arrange the first weighted values ​​of all first sampling points in descending order, and take the radial distance of the first sampling points corresponding to the first weighted values ​​in their respective three-dimensional spherical spaces. S223. Based on the two independent random variables introduced, both of which follow a uniform distribution in [0,1], calculate the azimuth and polar angle respectively, and determine a second sampling point by combining the radial distance obtained in step S222. S224. Repeat step S223 to sample multiple second sampling points at different radial distances.

7. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 6, characterized in that, In step S23, the formula for generating the final rendered value of the pixel corresponding to all sampling points using spherical rendering technology is as follows: , where P z r represents the final rendered value of the pixel corresponding to all sampling points. j ω represents the radial distance of the j-th sampling point within its three-dimensional spherical space. j,2 ω represents the second weighted value of the j-th sampling point. j,2 =T j,2 (1-exp(-σ j,2 (r j+1 -r j ))), T j,2 This represents the second transmittance at the j-th sampling point. , σ j,2 Let j represent the second volume density at the j-th sampling point, j=1,2,…,N c +N f -1 hour r j+1 This represents the radial distance of the (j+1)th sampling point within its three-dimensional spherical space, where j=N c +N f time r j+1 =r j , σ k,2 c represents the second volume density at the k-th sampling point. j,2 This represents the second color value of the j-th sampling point.

8. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 1, characterized in that, The model is trained using an image patch consisting of all pixels in the hole patch as the true label, and the network loss is calculated after each training round, which is a weighted sum of adaptive multi-scale structural similarity loss and adaptive mean square error loss.

9. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 8, characterized in that, The formula for calculating the adaptive multi-scale structural similarity loss is as follows: , where L AM-SSIM AM-SSIM(B) represents the adaptive multi-scale structural similarity loss, where Num represents the batch size for each training round. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t Multiscale structural similarity between them M represents the number of scales in the multiscale decomposition, bri M (B n,pred B n,g.t. ), con m (B n,pred B n,g.t. ), str m (B n,pred B n,g.t. ) corresponds to B n,pred With B n,g.t Structural similarity among luminance, contrast, and structural components at corresponding scales, α M β m τ m The corresponding learned weight parameters represent the brightness, contrast, and structure components at the corresponding scale.

10. The method for super-resolution reconstruction of neural radiation field volume for detecting cerebral microbleeds according to claim 9, characterized in that, The formula for calculating the adaptive mean square error loss is as follows: , where L A-MSE A-MSE(B) represents the adaptive mean squared error loss. n,pred B n,g.t. ) represents the nth reconstructed image patch B in a batch. n,pred With the corresponding real label B n,g.t The mean square error between them B represents B n,pred With B n,g.t The number of pixels in P b,g.t. B n,g.t The pixel value of the b-th pixel in the image, P b,z B n,pred The pixel value of the b-th pixel in the image, P b,s Indicates with B n,g.t Corresponding initial rendered image block B n,s,pred The pixel value of the b-th pixel in the image is given. The initial rendered image block is composed of the initial rendered values ​​of all pixels in the hole patch corresponding to a sample in the training set.

Images