A two-dimensional pseudo-color retinal nerve fiber layer thickness map artifact repairing method based on double-structure prior guidance and closed-loop coupling reconstruction

Abstract

The application discloses a two-dimensional pseudo-color retinal nerve fiber layer thickness map artifact repairing method based on double-structure prior guidance and closed-loop coupling reconstruction. The method splices an artifact-containing RNFLT image, an artifact mask, a main structure prior graph and a detail structure prior graph into a joint input, extracts features through a shared encoder, and respectively recovers a main structure, a detail structure and a repaired image by a main structure decoder, a detail structure decoder and a reconstruction decoder. The structure prediction result is fed back to the last stage of the reconstruction decoder, and consistency constraint is performed through structure extraction from the repaired image and structure branch output, so that closed-loop coupling is formed. Combined with a phased training strategy, the application can improve the structural integrity of large-area artifact repairing and the availability of glaucoma auxiliary analysis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, deep learning image restoration, and ophthalmic auxiliary diagnosis, and particularly to a method for repairing artifacts in two-dimensional pseudocolor retinal nerve fiber layer thickness maps based on dual-structure prior guidance and closed-loop coupling reconstruction. Background Technology

[0002] The retinal nerve fiber layer (RNFL) is a crucial innermost structure of the retina, primarily composed of retinal ganglion cell axons. Its thickness distribution is closely related to the development and progression of ophthalmic diseases such as glaucoma. Clinically, optical coherence tomography (OCT) is commonly used to scan the fundus of the subject, and the results are combined with retinal layer analysis to generate a two-dimensional pseudo-color retinal nerve fiber layer thickness map, i.e., a retinal nerve fiber layer thickness (RNFLT) image, reflecting the thickness distribution characteristics of the nerve fiber layer in different regions. Since diseases such as glaucoma are often accompanied by RNFL thinning, defects, and structural abnormalities, image analysis based on the two-dimensional pseudo-color retinal nerve fiber layer thickness map is of great significance for glaucoma screening, diagnostic reference, and progression assessment.

[0003] However, during actual OCT acquisition, black artifacts may appear in the generated two-dimensional pseudo-color retinal nerve fiber layer thickness map due to factors such as subject eye movement, blinking, changes in tear film state, signal attenuation, scan registration deviation, and equipment noise. These artifacts disrupt the original spatial continuity and structural integrity of the thickness map, leading to inaccurate local thickness distribution representation. This affects the doctor's observation and judgment of lesion areas and reduces the reliability of subsequent image analysis results.

[0004] Existing methods for artifact removal in medical images mostly employ image interpolation, filtering enhancement, or convolutional neural network-based image inpainting to fill in localized defects. While these methods are effective for natural images or small-scale defect repair tasks, they still have limitations for medical images with clearly defined structural distribution patterns, such as two-dimensional pseudocolor retinal nerve fiber layer thickness maps. Especially when artifacts are large or located in critical structural regions, existing methods are prone to problems such as overly smoothed repair areas, local structural breaks, and unnatural integration with surrounding normal areas.

[0005] Furthermore, the neural fiber structure in the two-dimensional pseudocolor retinal neural fiber layer thickness map exhibits a certain hierarchy, including both relatively continuous main structures and locally divergent, weak fibers. Existing image inpainting methods typically focus more on pixel completion of the missing areas or overall texture restoration, failing to adequately distinguish and utilize structural information at different levels. Moreover, even when structural prediction is introduced in some methods, the structural information is often used as intermediate supervision or auxiliary output, lacking feedback constraints between the prediction and the final image reconstruction branch. This results in limited direct guidance of the structural prediction results on the final inpainted image, leading to inconsistencies between the structural prediction and the final reconstruction results.

[0006] Two-dimensional pseudocolor retinal nerve fiber layer thickness maps not only contain color, texture, and morphological information at the image level, but also reflect the spatial distribution characteristics of retinal nerve fiber layer thickness in different regions. If the restoration process mainly relies on pixel-level reconstruction errors and lacks constraints on structural characterization and analysis features, the restoration result, while visually smooth, may weaken information related to subsequent analysis, such as local thickness distribution or structural continuity. Therefore, it is still necessary to propose an artifact restoration method for two-dimensional pseudocolor retinal nerve fiber layer thickness maps to improve the structural continuity of the restoration result under large-area artifact conditions and provide a more reliable image basis for subsequent glaucoma-related image analysis. Summary of the Invention

[0007] The purpose of this invention is to provide a two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair method based on dual-structure prior guidance and closed-loop coupling reconstruction, so as to solve the problems of existing two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair methods, which are prone to main structure breakage, loss of fine fiber details, unnatural artifact boundary connection and insufficient structural consistency of repair results under large-area black artifact conditions.

[0008] To achieve the above objectives, the present invention provides a method for repairing artifacts in two-dimensional pseudocolor retinal nerve fiber layer thickness maps based on dual-structure prior guidance and closed-loop coupling reconstruction, the method comprising a training phase and a repair phase.

[0009] The training phase includes:

[0010] S1. Obtain a training sample of a two-dimensional pseudo-color retinal nerve fiber layer thickness map, wherein the training sample includes a three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map with black artifacts and a corresponding reference two-dimensional pseudo-color retinal nerve fiber layer thickness map without artifacts.

[0011] S2. Generate an artifact binary mask based on the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts, and generate a trunk structure prior map and a detail structure prior map based on the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts and the artifact binary mask; wherein, the trunk structure prior map is used to characterize the continuous structure of the trunk of the retinal nerve fiber layer, and the detail structure prior map is used to characterize the thin and weak fiber structure;

[0012] S3. The three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts, the artifact binary mask, the trunk structure prior map and the detail structure prior map are spliced ​​together to obtain a six-channel joint input.

[0013] S4. The six-channel joint input is fed into a single encoder-three decoder artifact restoration network, which includes a shared gated convolutional encoder, a backbone structure decoder, a detail structure decoder, and a reconstruction decoder, and the shared features, backbone structure prediction map, detail structure prediction map, and restored image are obtained.

[0014] S5. In the final decoding stage of the reconstruction decoder, the backbone structure prediction map and the detail structure prediction map are fused with the reconstruction decoding features as structural feedback information to obtain the fused reconstruction features, and a repaired image is generated based on the fused reconstruction features.

[0015] S6. The artifact restoration network is trained using a joint loss, which includes basic reconstruction loss, structural coupling loss, boundary ring gradient consistency loss, and multi-layer diagnostic feature consistency loss. The structural coupling loss is used to constrain the trunk structure representation and detail structure representation obtained in the final restored image to be consistent with the structural branch output.

[0016] The repair phase includes: generating a corresponding artifact binary mask, a trunk structure prior map, and a detail structure prior map for the two-dimensional pseudo-color retinal nerve fiber layer thickness map to be repaired, and performing channel splicing with the image to be repaired to form a six-channel joint input; feeding the six-channel joint input into the trained artifact repair network, and outputting the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0017] Further, the artifact binary mask, the trunk structure prior map, and the detail structure prior map are generated as follows: the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts is grayscaled, and the black artifact region is extracted according to a preset threshold to obtain an initial artifact mask; morphological closing operation and hole filling are performed on the initial artifact mask to obtain an artifact binary mask; based on the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts and the artifact binary mask, the trunk structure prior map is obtained through a trunk structure extraction operator; based on the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts, the artifact binary mask, and the trunk structure prior map, the detail structure prior map is obtained through a detail structure extraction operator.

[0018] Furthermore, the trunk structure extraction operator includes one or more of local contrast enhancement response, multi-scale bright structure response, and multi-scale ridge line response; the detail structure extraction operator includes one or more of fine-scale ridge line response extraction, structural tensor consistency response extraction, trunk leakage suppression, and connected component screening. Through the above methods, the continuous trunk structure and weak fiber structure in the two-dimensional pseudo-color retinal nerve fiber layer thickness map are expressed in a layered manner.

[0019] Furthermore, the feature extraction network used in the multi-layer diagnostic feature consistency loss includes a convolutional neural network branch and a visual self-supervised feature extraction branch. The convolutional neural network branch is an EfficientNet series network or a ResNet series network, and the visual self-supervised feature extraction branch is a DINO series network, a visual Transformer network, or a medical image self-supervised pre-trained network. The feature extraction network is used to extract multi-layer feature representations from the restored image and the reference artifact-free image to constrain the restored image to maintain structural discrimination information related to the thickness distribution of retinal nerve fiber layers and glaucoma auxiliary analysis.

[0020] Furthermore, the single-encoder-three-decoder artifact inpainting network includes a shared encoder, a backbone structure decoder, a detail structure decoder, and a reconstruction decoder. The shared encoder includes multi-layer gated convolutional coding units, each comprising a feature convolution branch and a gated convolution branch, used to suppress invalid responses in artifact regions while extracting image features. The bottleneck layer of the shared encoder incorporates a multi-receptive-field feature fusion module to fuse local structural features, mesoscale structural features, and larger receptive-field contextual features. The final decoding stage of the reconstruction decoder incorporates a structure feedback gated fusion layer to fuse the final reconstruction decoding features, the backbone structure prediction map, and the detail structure prediction map to generate the final inpainted image.

[0021] Furthermore, the artifact restoration network is trained using a phased training strategy. This phased training strategy includes: a first phase, training the reconstruction decoder and the backbone structure decoder to enable the reconstruction decoder to acquire basic artifact restoration capabilities and the backbone structure decoder to acquire backbone continuous structure recovery capabilities; a second phase, jointly training the reconstruction decoder, backbone structure decoder, and detail structure decoder, and opening up structural feedback information through gradient backpropagation from the reconstruction decoder to jointly optimize the backbone structure prediction map, detail structure prediction map, and the final restored image; and a third phase, based on the joint training, introducing multi-layer diagnostic feature consistency loss for fine-tuning, so that the restored two-dimensional pseudocolor retinal nerve fiber layer thickness map maintains glaucoma-related auxiliary analysis features while restoring visual structure.

[0022] Furthermore, the joint loss function is:

[0023] ;

[0024] in, Based on the loss of reconstruction, For structural coupling loss, For boundary ring gradient consistency loss, For multi-level diagnostic feature consistency loss, , , , For the corresponding weights;

[0025] Furthermore, the basic reconstruction loss is used to constrain the pixel differences between the restored image and the reference artifact-free image, and assigns different weights to artifact regions and effective regions; the structural coupling loss is used to constrain the backbone structure representation and detail structure representation obtained in the final restored image to be consistent with the backbone structure prediction map and detail structure prediction map, respectively; the boundary ring gradient consistency loss is used to constrain the consistency of gradient magnitude and gradient direction near the boundary of the artifact region; the multilayer diagnostic feature consistency loss is used to constrain the feature representation of the restored image and the reference artifact-free image to be consistent in multiple preset layers of the diagnostic feature extraction network.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects:

[0027] First, the present invention uses a six-channel joint input consisting of a binary mask for artifacts, a priori maps of the backbone structure and a priori maps of the detail structure, to enable the artifact restoration network to simultaneously perceive artifact regions, continuous backbone structures and weak fiber structures, thereby improving the structured restoration capability under conditions of large-area black artifacts.

[0028] Second, the present invention adopts a single encoder-three decoder structure, which recovers the main continuous structure, the weak fiber structure and the final repaired image respectively on the basis of shared coding features, so that the image repair process is guided by the coordinated structure branches and the detail structure branches.

[0029] Third, the present invention feeds back the main structure prediction map and the detailed structure prediction map to the final stage of the reconstruction decoder, and constrains the structural representation obtained from the final repaired image to be consistent with the structural branch output through structural coupling loss, so that a coupling relationship of structural prediction, structural feedback, image reconstruction and consistency constraint is formed between the structural branch output and the repaired image, thereby improving the structural consistency of the repair result.

[0030] Fourth, the present invention introduces gradient consistency constraints within the artifact boundary ring, which helps to reduce color abruptness, edge breakage, and structural orientation disorder at the artifact boundary, and improves the natural connection effect between the repaired area and the surrounding effective area.

[0031] Fifth, this invention optimizes the basic reconstruction capability, backbone structure recovery capability, detailed structure recovery capability, and diagnostic-related feature preservation capability through a phased training strategy, which can reduce gradient interference in the early stage of multi-branch network training and improve model training stability.

[0032] Sixth, this invention repairs images by constraining the loss of consistency of multi-layer diagnostic features, maintaining structural discrimination information related to the distribution of retinal nerve fiber layer thickness and glaucoma auxiliary analysis. This makes the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map more suitable for subsequent glaucoma-related image analysis, auxiliary screening, or structural assessment while maintaining the integrity of visual structure. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall process of the two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair method based on dual-structure prior guidance and closed-loop coupling reconstruction of the present invention.

[0034] Figure 2 This is a schematic diagram of data construction and joint input for a two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair method based on dual-structure prior guidance and closed-loop coupling reconstruction according to the present invention.

[0035] Figure 3 This is a schematic diagram of a dual-structure priori-guided closed-loop coupled artifact repair network structure for a single encoder-three decoder according to the present invention.

[0036] Figure 4 This is a schematic diagram of the phased training and joint loss constraint of a two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair method based on dual-structure prior guidance and closed-loop coupling reconstruction according to the present invention.

[0037] Figure 5 This is a schematic diagram illustrating the closed-loop coupling relationship between the main structural branches, detailed structural branches, and reconstruction branches of this invention.

[0038] Figure 6 This is a schematic diagram of the artifact region and its corresponding artifact mask, trunk structure prior map and detail structure prior map in the two-dimensional pseudocolor retinal nerve fiber layer thickness map of the present invention. From left to right, they are the original RNFLT image, the artifact binary mask, the image containing artifacts, the trunk structure prior map and the detail structure prior map.

[0039] Figure 7 From left to right, the images are: an image with artifacts, the result of partial convolution method restoration, the result of gated convolution method restoration, the result of the method of this invention restoration, and a reference image without artifacts. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0041] This invention provides a method for artifact repair based on two-dimensional pseudo-color retinal nerve fiber layer thickness map reconstruction using dual-structure prior guidance and closed-loop coupling. This method uses a two-dimensional pseudo-color retinal nerve fiber layer thickness map as the processing object, addressing issues such as black artifacts, local defects, and structural discontinuities in the image. It constructs a binary artifact mask, a trunk structure prior map, and a detail structure prior map, and uses these, along with the image containing artifacts, as the joint input to the artifact repair network.

[0042] In terms of network structure, this embodiment of the invention employs a single-encoder-three-decoder artifact inpainting network. A shared encoder extracts shared features from the joint input, and the backbone structure decoder, detail structure decoder, and reconstruction decoder respectively obtain the backbone structure prediction map, detail structure prediction map, and the thickness map of the repaired two-dimensional pseudo-color retinal nerve fiber layer. Furthermore, the backbone structure prediction map and detail structure prediction map are introduced as structural feedback information into the final decoding stage of the reconstruction decoder, allowing the structural branch outputs to participate in the generation of the final repaired image. Network optimization is then achieved by combining structural consistency constraints and a staged training strategy.

[0043] The following combination Figures 1 to 7 The specific embodiments of the present invention will be described in detail below.

[0044] Example 1: Overall Process Example

[0045] refer to Figure 1This embodiment provides an overall workflow for a two-dimensional pseudocolor retinal nerve fiber layer thickness map artifact repair method. The method includes seven steps: image data acquisition, artifact masking and dual-structure prior construction, training sample construction and data augmentation, artifact repair network construction, structural feedback closed-loop coupling reconstruction, joint loss and phased training optimization, and artifact repair output.

[0046] In step S101, a two-dimensional pseudo-color retinal nerve fiber layer thickness map is obtained. Specifically, optical coherence tomography (OCT) equipment can be used to acquire retinal tomographic image data of the subject's fundus, and the retinal nerve fiber layer thickness information can be calculated based on the retinal layering results to generate a two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0047] In step S102, an artifact binary mask, a trunk structure prior map, and a detail structure prior map are generated. Specifically, the two-dimensional pseudo-color retinal nerve fiber layer thickness map containing artifacts is preprocessed, and an artifact binary mask is generated based on the grayscale features of the black artifact regions in the image. Based on the two-dimensional pseudo-color retinal nerve fiber layer thickness map and the artifact binary mask, a trunk structure prior map and a detail structure prior map are generated. The trunk structure prior map is used to characterize the continuous structure of the retinal nerve fiber layer trunk, and the detail structure prior map is used to characterize the thin and weak fiber structure.

[0048] In step S103, training samples are constructed and data augmentation is performed. Specifically, during the training phase, a binary mask of artifacts is overlaid on a reference artifact-free two-dimensional pseudo-color retinal nerve fiber layer thickness map to construct paired training samples corresponding to the artifact-containing image and the reference artifact-free image. Augmented training data is generated by random rotation, random translation, or random artifact overlay.

[0049] In step S104, a single-encoder-three-decoder artifact repair network is constructed. The artifact repair network includes a shared encoder, a backbone structure decoder, a detail structure decoder, and a reconstruction decoder. The shared encoder is used to extract features from the joint input to obtain shared representation features; the backbone structure decoder is used to output a backbone structure prediction map; the detail structure decoder is used to output a detail structure prediction map; and the reconstruction decoder is used to output a repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0050] In step S105, structural feedback closed-loop coupled reconstruction is performed. Specifically, in the final decoding stage of the reconstruction decoder, the backbone structure prediction map and detail structure prediction map are used as structural feedback information and fused with the reconstruction decoding features to obtain structurally enhanced reconstruction features. The final restored image is then generated based on these structurally enhanced reconstruction features. In this way, the structural branch output is no longer just an auxiliary supervision result, but participates in the final image reconstruction process.

[0051] In step S106, the artifact repair network is optimized using a joint loss and a phased training strategy. The joint loss includes basic reconstruction loss, structural coupling loss, boundary ring gradient consistency loss, and multi-layer diagnostic feature consistency loss. During training, the basic reconstruction capability and backbone structure recovery capability are optimized first, then detailed structural branches and structural feedback paths are introduced for joint training, and finally, feature consistency constraints are introduced for fine-tuning as needed.

[0052] In step S107, a corresponding artifact binary mask, trunk structure prior map, and detail structure prior map are generated for the two-dimensional pseudo-color retinal nerve fiber layer thickness map to be repaired. These are then channel-stitched with the image to be repaired and input into the trained artifact repair network to output the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0053] Example 2: Artifact Masking and Dual-Structure Prior Construction Example

[0054] refer to Figure 2 and Figure 6 This embodiment illustrates the construction methods of artifact binary mask, backbone structure prior map, detail structure prior map, and six-channel joint input.

[0055] In this embodiment, a two-dimensional pseudo-color retinal nerve fiber layer thickness map is first obtained. This map can be obtained from retinal tomographic images acquired by an optical coherence tomography (OCT) device after retinal nerve fiber layer thickness calculation and pseudo-color mapping. During the training phase, a binary mask of artifacts can be overlaid onto a reference artifact-free two-dimensional pseudo-color retinal nerve fiber layer thickness map to construct a three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts. During the repair phase, the artifact-containing two-dimensional pseudo-color retinal nerve fiber layer thickness map to be repaired is directly used as the input image.

[0056] Specifically, the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts is grayscaled, and the black artifact region is extracted according to the preset grayscale threshold to obtain the initial artifact mask.

[0057] ;

[0058] Subsequently, morphological closing operations and hole filling are performed on the initial artifact mask to enhance the connectivity of the artifact region and remove local small holes, resulting in a binary artifact mask.

[0059] ;

[0060] The artifact binary mask is used to identify invalid regions and regions to be repaired in the image.

[0061] After obtaining the artifact binary mask, based on the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts and the artifact binary mask, the trunk structure prior map and the detail structure prior map are constructed respectively.

[0062] The trunk structure prior map is used to characterize the continuous trunk structure in the retinal nerve fiber layer. In one specific implementation, local contrast enhancement is first performed on the effective region to obtain a local bright structure response; then, multi-scale filtering is used to obtain the continuous trunk structure response; subsequently, the multi-scale ridge line response is fused and normalized to obtain the trunk structure prior map. , .

[0063] The detailed structure prior map is used to characterize the delicate fiber structures in the retinal nerve fiber layer. In one specific implementation, the fine-scale ridge response is first extracted, then the structural tensor consistency response is used to filter out oriented unstable regions, followed by trunk leakage suppression to reduce the interference of the trunk structure on the detailed structure, and finally, connected component filtering is used to remove isolated noise to obtain the detailed structure prior map. , .

[0064] like Figure 6 As shown, the trunk structure prior map and the detail structure prior map respectively reflect different levels of structural information in the two-dimensional pseudocolor retinal nerve fiber layer thickness map. The trunk structure prior map focuses on the continuous main bundle structure, while the detail structure prior map focuses on the weak fibers and local detail structures. Together, they provide structural constraint information for the subsequent artifact repair network.

[0065] like Figure 2 As shown, the three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts, the artifact binary mask, the trunk structure prior map, and the detail structure prior map are concatenated to form the joint input of the artifact repair network. When the two-dimensional pseudo-color retinal nerve fiber layer thickness map is a three-channel pseudo-color image, the joint input is a six-channel input. , .

[0066] The six-channel joint input simultaneously contains image color thickness information, artifact location information, backbone continuity structure information, and detail fiber structure information, which are used to input the subsequent single encoder-triple decoder artifact repair network.

[0067] In this way, this embodiment can explicitly distinguish between artifact regions, continuous trunk structures, and weak fiber structures during the input stage, providing a structural prior basis for the collaborative repair of subsequent trunk structure decoders, detail structure decoders, and reconstruction decoders.

[0068] Example 3: Single Encoder-Three Decoder Artifact Repair Network Example

[0069] refer to Figure 3 This embodiment illustrates the structure of a single encoder-three decoder artifact restoration network. The artifact restoration network receives the six-channel combined input formed in Embodiment 2 and outputs a backbone structure prediction map, a detail structure prediction map, and a restored two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0070] like Figure 3 As shown, the artifact restoration network includes a shared encoder, a backbone decoder, a detail decoder, and a reconstruction decoder. The shared encoder is used to extract features from the six-channel joint input, obtaining shared representation features containing pseudocolor image information, artifact location information, backbone structure information, and detail structure information. , The trunk structure decoder is used to recover the continuous trunk structure of the retinal nerve fiber layer based on the shared representation features, and outputs a trunk structure prediction map. The detail structure decoder is used to recover the weak fiber structure and local texture details based on the shared representation features, and outputs a detail structure prediction map. , .

[0071] In one specific embodiment, the shared encoder may include multi-level downsampling feature extraction units. Each downsampling feature extraction unit may include convolutional layers, normalization layers, and nonlinear activation layers to progressively extract multi-scale features from low-level textures to high-level structures. To accommodate the irregular shapes, discontinuous boundaries, and uneven distribution of effective regions in black artifact regions, the convolutional layers in the shared encoder may employ gated convolutional coding units. The gated convolutional coding unit includes a feature convolution branch and a gated convolution branch. The feature convolution branch generates candidate feature responses, and the gated convolution branch generates spatial gating weights, thereby suppressing invalid responses in artifact regions and enhancing the feature representation of effective structural regions. , .

[0072] In one specific implementation, the bottleneck layer of the shared encoder may be equipped with a multi-receptive-field feature fusion module. This module includes multiple parallel convolutional branches, used to extract local structural features, mesoscale structural features, and larger-scale contextual features, respectively. The outputs of these parallel convolutional branches are concatenated and fused to obtain the bottleneck fused features. Through this structure, the network can simultaneously perceive local details, continuous backbone structure, and thickness distribution trends over a large area, thereby improving the structural recovery capability within large artifact regions.

[0073] In one specific implementation, the backbone decoder, detail decoder, and reconstruction decoder can all employ a multi-level upsampling recovery structure corresponding to the shared encoder. During the upsampling process, each decoder can receive skip connection features of the same scale from the shared encoder to recover spatial resolution and preserve low-level texture information. Specifically, the backbone decoder focuses on recovering the continuous main bundle structure, the detail decoder focuses on recovering the thin fiber structure, and the reconstruction decoder focuses on recovering the color distribution, thickness distribution, and local structural continuity of the final pseudo-color image.

[0074] In one specific embodiment, a structural feedback fusion layer is provided in the final decoding stage of the reconstruction decoder. This structural feedback fusion layer receives reconstruction decoding features, a backbone structure prediction map, and a detail structure prediction map, and fuses the above information to generate the final restored image. , The specific closed-loop coupling method of the structural feedback fusion layer will be discussed in Example 4. Figure 5 Further explanation.

[0075] Through the aforementioned single encoder-three decoder structure, this embodiment can model the main continuous structure, the detailed fiber structure, and the final repaired image separately based on shared representations. This allows the artifact repair process to no longer rely solely on a single image reconstruction branch, but is simultaneously guided by both the main structure branch and the detailed structure branch, thereby improving the structured repair capability of the two-dimensional pseudocolor retinal nerve fiber layer thickness map under conditions of large-area black artifacts.

[0076] Example 4: Structural Feedback Closed-Loop Coupling Example

[0077] refer to Figure 5 This embodiment illustrates the structural feedback closed-loop coupling relationship between the main structural branches, detailed structural branches, and reconstruction branches.

[0078] In this embodiment, the trunk structure decoder outputs a trunk structure prediction map, and the detail structure decoder outputs a detail structure prediction map. The trunk structure prediction map is used to characterize the continuous trunk structure in the two-dimensional pseudo-color retinal nerve fiber layer thickness map, and the detail structure prediction map is used to characterize the thin fiber structure and local detail structure. Unlike methods that only use structural branches as auxiliary supervision branches, this embodiment uses the trunk structure prediction map and detail structure prediction map as structural feedback information, inputting them into the final decoding stage of the reconstruction decoder, so that the structural branch output directly participates in the generation of the final restored image.

[0079] Specifically, after obtaining the reconstructed decoding features in the final decoding stage, the reconstruction decoder fuses these features, the backbone structure prediction map, and the detail structure prediction map to obtain the structurally enhanced reconstruction features. The fusion method can be one or more combinations of channel stitching, weighted summation, gated fusion, attention modulation, or residual injection. In one specific embodiment, the reconstructed decoding features, backbone structure prediction map, and detail structure prediction map can be channel stitched and then input into the structure feedback gated fusion layer. The contribution of backbone structure information and detail structure information to the final restored image can be adjusted by the gate weights.

[0080] Furthermore, to ensure structural consistency in the final restored image, a differentiable structure extraction operator is applied to the restored 2D pseudo-color retinal nerve fiber layer thickness map during training to re-extract the trunk structure representation and detail structure representation in the restored image. Consistency constraints are applied between the trunk structure prediction map output by the trunk structure decoder and the detail structure prediction map output by the detail structure decoder.

[0081] In other words, the closed-loop coupling relationship in this embodiment includes two directions: on the one hand, the main structure prediction map and the detail structure prediction map participate in the image generation process of the reconstruction branch as structural feedback information; on the other hand, after the final repaired image is extracted from the differentiable structure, it is aligned with the output of the structural branch. This forms a closed-loop coupling mechanism of "structural prediction - structural feedback - image reconstruction - structural re-extraction - consistency constraint".

[0082] Through the aforementioned structural feedback closed-loop coupling method, the reconstruction decoder can be guided by both the continuous trunk structure and the weak fiber structure when generating the repaired image, thereby reducing the possibility of trunk breakage, edge misalignment, loss of detail, and unnatural connection between the repaired area and the surrounding effective area in large black artifact areas.

[0083] Example 5: Phased Training and Joint Loss Example

[0084] refer to Figure 4 This embodiment illustrates the phased training strategy and joint loss constraint method of the artifact restoration network.

[0085] In this embodiment, to reduce the multi-branch gradient interference in the early stages of training of the single encoder-three decoder structure, and to gradually enable the reconstruction branch, backbone branch, and detail branch to acquire stable representation capabilities, the artifact inpainting network is optimized using a phased training strategy. This phased training strategy includes a basic inpainting and backbone structure-priority training phase, a joint training phase involving detail structure addition and structural feedback, and a diagnostically friendly fine-tuning phase.

[0086] The first phase is the basic repair and trunk structure priority training phase. In this phase, the shared encoder, reconstruction decoder, and trunk structure decoder are trained preferentially. This enables the reconstruction decoder to acquire basic artifact repair capabilities, and the trunk structure decoder to recover the continuous trunk structure in the retinal nerve fiber layer. This phase mainly uses basic reconstruction loss, trunk structure supervision loss, and boundary ring gradient consistency loss for constraints to reduce trunk structure breaks and boundary discontinuities in large black artifact regions.

[0087] The second stage is the joint training stage of adding detailed structure and structural feedback. Based on the training results of the first stage, a detailed structure decoder is added, enabling it to learn weak fiber structures and local texture details. In this stage, the backbone structure prediction map output by the backbone structure decoder, the detailed structure prediction map output by the detailed structure decoder, and the final-level decoded features of the reconstruction decoder are input into the structural feedback fusion layer, allowing for joint optimization of the backbone structure prediction map, detailed structure prediction map, and the final restored image. To reduce the instability caused by the structural branches on the reconstruction branches in the early stages of training, gradient separation can be performed on some structural feedback paths in the early part of the second stage, followed by gradually opening up gradient backpropagation of the structural feedback paths.

[0088] The third stage is the diagnostic-friendly fine-tuning stage. Building upon the joint training in the second stage, a multi-layer diagnostic feature consistency loss is introduced. The restored 2D pseudo-color retinal nerve fiber layer thickness map and a reference artifact-free 2D pseudo-color retinal nerve fiber layer thickness map are input into the diagnostic feature extraction network. Diagnostic-related features are extracted at multiple preset levels, and the differences between the two maps in the diagnostic feature space are constrained. Through fine-tuning in this stage, the restored image maintains the continuity of visual structure and the authenticity of details while preserving as much thickness distribution features and discriminative information as possible that are relevant to glaucoma auxiliary analysis.

[0089] In this embodiment, the joint loss includes basic reconstruction loss, structural coupling loss, boundary ring gradient consistency loss, and multi-layer diagnostic feature consistency loss, and its total loss function can be expressed as:

[0090] ;

[0091] in, Based on the loss of reconstruction, For structural coupling loss, For boundary ring gradient consistency loss, For multi-level diagnostic feature consistency loss, , , , For the corresponding weights.

[0092] The basic reconstruction loss This method constrains the pixel differences between the repaired image and the reference image without artifacts, and can assign different weights to the artifact regions and the effective regions, allowing the network to focus on learning the completion ability of the artifact regions while maintaining the image consistency of the effective regions. .

[0093] The structural coupling loss This loss is used to constrain the consistency between the final restored image and the structural branch output. Specifically, a differentiable backbone structure extraction operator and a differentiable detail structure extraction operator are applied to the restored image to obtain the backbone structure representation and detail structure representation in the restored image, respectively. These representations are then constrained to consistency with the predicted backbone structure map and the predicted detail structure map. This loss ensures that the final restored image itself possesses backbone continuity and detail texture distribution consistent with the structural branch output. .

[0094] The boundary ring gradient consistency loss This is used to constrain structural continuity near the boundaries of artifact regions. Specifically, a boundary ring can be constructed based on the dilated and eroded regions of the artifact binary mask. Within this boundary ring, the gradient magnitude and gradient direction of the restored image are constrained to be consistent with the reference artifact-free image, thereby reducing color abrupt changes and boundary breaks at artifact edges. .

[0095] The consistency loss of multi-layer diagnostic features This constraint is used to maintain glaucoma diagnostic information in the restored image. The diagnostic feature extraction network may include a convolutional neural network branch and a visually self-supervised feature extraction branch, used to extract multi-layer diagnostic features from the restored image and a reference artifact-free image, and calculate the differences between corresponding layer features. This loss constraint reduces the disruption of glaucoma-related thickness distribution features during the restoration process. .

[0096] Through the aforementioned phased training strategy and joint loss constraints, the artifact repair network can be progressively optimized in the order of basic repair and trunk structure restoration, detailed structure learning and structural feedback joint optimization, and preservation of diagnostic-related features. This improves the structural integrity, boundary naturalness, and diagnostic usability of the two-dimensional pseudocolor retinal nerve fiber layer thickness map under conditions of large-area black artifacts.

[0097] Example 6: Artifact Repair Output and Effect Verification Example

[0098] refer to Figure 7 This embodiment illustrates the artifact restoration output process and the method for verifying the restoration effect after training is completed.

[0099] In this embodiment, for the two-dimensional pseudo-color retinal nerve fiber layer thickness map to be repaired, the corresponding artifact binary mask, trunk structure prior map, and detail structure prior map are first generated according to the method described in Embodiment 2. Then, the image to be repaired, the artifact binary mask, the trunk structure prior map, and the detail structure prior map are concatenated to form a six-channel joint input. Finally, the six-channel joint input is fed into the trained single encoder-three decoder artifact repair network to obtain the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map.

[0100] Specifically, the trained artifact repair network does not need to refer to the artifact-free image as input during the inference phase. It completes the artifact region completion only based on the image to be repaired and the structural prior information generated from the image to be repaired. The trunk structure prior map is used to guide the network to restore the continuous structure of the trunk of the retinal nerve fiber layer, the detail structure prior map is used to guide the network to restore the local thin fiber structure, and the artifact binary mask is used to indicate the area to be repaired, so that the network can perform targeted structural reconstruction within the artifact region.

[0101] like Figure 7 As shown, in one experimental embodiment, a two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts was input into different artifact repair methods, and the repair results of each method were compared with a reference image without artifacts. Figure 7 The image shows an image with artifacts, the results of restoration using partial convolution methods, the results of restoration using gated convolution methods, the results of restoration using the method in this embodiment, and a reference image without artifacts.

[0102] from Figure 7 As can be seen, compared to restoration methods that rely solely on a single reconstruction branch, the method in this embodiment can better maintain the continuity of the main structure of the retinal nerve fiber layer within large areas of black artifacts, and improve the color transition and structural connection at artifact boundaries. Furthermore, because this embodiment introduces a priori maps of the main structure and detail structure during the input stage, and sets up a closed-loop coupling mechanism for main structure branches, detail structure branches, and structural feedback within the network, the restoration result can maintain good structural hierarchy consistency while completing pixels.

[0103] Furthermore, the repaired two-dimensional pseudocolor retinal nerve fiber layer thickness map can be used as input data for subsequent glaucoma-assisted screening, classification, or disease assessment. In one specific implementation, the repaired image can be input into a diagnostic feature extraction network or a glaucoma-assisted analysis model to evaluate the impact of the repair results on the preservation of diagnostic-related features.

[0104] Through the above-described repair output and effect verification methods, it can be demonstrated that the method of this embodiment, while removing black artifacts from the two-dimensional pseudocolor retinal nerve fiber layer thickness map, helps to maintain the natural transition relationship between the main continuous structure, the detailed fiber structure, and the artifact boundary.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the image preprocessing method, artifact mask generation method, backbone structure prior extraction operator, detail structure prior extraction operator, number of network layers, convolution form, upsampling method, structural feedback fusion method, loss function weights, and training parameters in the above embodiments can be adjusted or replaced without departing from the concept of the present invention.

[0106] The method described in the embodiments of the present invention can be implemented by software, hardware, or a combination of software and hardware. Accordingly, the present invention can also be embodied as an electronic device, the electronic device including a processor, a memory, and a computer program stored in the memory and executable by the processor; when the computer program is executed by the processor, the two-dimensional pseudo-color retinal nerve fiber layer thickness map artifact repair method described in the above embodiments is implemented.

[0107] In summary, this invention uses a binary mask for artifacts, a priori maps of the main structure and details to form a joint input, and utilizes a single encoder-three decoder artifact restoration network to achieve the coordinated restoration of the main structure, details, and the final restored image. Furthermore, through structural feedback closed-loop coupling, a phased training strategy, and joint loss constraints, it improves the structural integrity, boundary continuity, and diagnostic feature preservation capabilities of the two-dimensional pseudocolor retinal nerve fiber layer thickness map under conditions of large-area black artifacts.

Claims

1. A method for repairing artifacts in two-dimensional pseudocolor retinal nerve fiber layer thickness maps based on dual-structure prior guidance and closed-loop coupling reconstruction, characterized in that, Includes a training phase and a repair phase; The training phase includes: S1. Obtain training samples of two-dimensional pseudo-color retinal nerve fiber layer thickness maps, wherein the training samples include three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness maps containing black artifacts. And its corresponding artifact-free 2D pseudocolor retinal nerve fiber layer thickness map ; S2. Based on the aforementioned three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts. Generate artifact binary mask Based on the three-channel two-dimensional pseudocolor retinal nerve fiber layer thickness map containing black artifacts, and the artifact binary mask Generate a priori diagram of the trunk structure and detailed structural prior diagram ; wherein, the a priori diagram of the main structure The prior art diagram of the detailed structure is used to characterize the continuous structure of the main trunk of the retinal nerve fiber layer. Used to characterize the structure of fine and weak fibers; S3. The three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts, the artifact binary mask, the trunk structure prior map, and the detail structure prior map are concatenated to obtain a six-channel joint input: ； S4. Input the six channels together. The artifact repair network is fed into a single encoder-three decoder network, the artifact repair network including a shared gated convolutional encoder. Backbone Decoder Detail structure decoder and rebuild decoder and obtain shared features Main structure prediction diagram Detailed structure prediction diagram and image restoration : ； S5. In the final decoding stage of the reconstruction decoder, the backbone structure prediction diagram is... and detailed structure prediction map As structural feedback information and reconstruction decoding features Fusion: ； in, Represents structural feedback fusion mapping, Indicates the reconstructed features after fusion; S6. The artifact restoration network is trained using a joint loss function, which is: ； in, Based on the loss of reconstruction, For structural coupling loss, For boundary ring gradient consistency loss, For multi-level diagnostic feature consistency loss, , , , For the corresponding weights; The repair phase includes: For the two-dimensional pseudo-color retinal nerve fiber layer thickness map to be repaired, a corresponding artifact binary mask, trunk structure prior map, and detail structure prior map are generated. These are then concatenated with the image to be repaired to form a six-channel joint input. The six-channel joint input is then fed into the trained artifact repair network, and the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map is output.

2. The method for repairing artifacts in a two-dimensional pseudocolor retinal nerve fiber layer thickness map according to claim 1, characterized in that, The artifact binary mask Prior diagram of the main structure and detailed structural prior diagram Generate as follows: Thickness map of the three-channel two-dimensional pseudocolor retinal nerve fiber layer containing black artifacts. Perform grayscale conversion to obtain a grayscale image. And according to the preset threshold Extract the black artifact regions to obtain the initial artifact mask: ； For the initial artifact mask Perform morphological closing operations and hole filling to obtain the artifact binary mask. : ； in, The structure element is indicated as Morphological closing operation, This indicates a hole-filling operation; Based on the aforementioned three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts and the artifact binary mask By extracting the backbone structure operator, the prior graph of the backbone structure is obtained. : ； in, This represents the backbone structure extraction operator, which includes local contrast enhancement response, multi-scale bright structure response, and multi-scale ridge line response. This indicates element-wise multiplication; Based on the aforementioned three-channel two-dimensional pseudo-color retinal nerve fiber layer thickness map containing black artifacts The artifact binary mask and the aforementioned trunk structure prior diagram By using the detail structure extraction operator, a detail structure prior graph is obtained. : ； in, The detailed structure extraction operator includes fine-scale ridge response extraction, structural tensor consistency response extraction, trunk leakage suppression, and connected component filtering.

3. The method for repairing artifacts in a two-dimensional pseudocolor retinal nerve fiber layer thickness map according to claim 1, characterized in that, The diagnostic feature extraction network includes a convolutional neural network branch and a visual self-supervised feature extraction branch. The convolutional neural network branch is an EfficientNet series network or a ResNet series network, and the visual self-supervised feature extraction branch is a DINO series network, a visual Transformer network, or a medical image self-supervised pre-trained network.

4. The method for repairing artifacts in a two-dimensional pseudocolor retinal nerve fiber layer thickness map according to claim 1, characterized in that, The U-NET artifact restoration network with a single encoder-three decoder structure specifically includes a shared encoder, a backbone decoder, a detail decoder, and a reconstruction decoder. The shared encoder comprises four gated convolutional coding units, each including a feature convolution branch and a gated convolution branch. The feature convolution branch uses 3×3 convolutions to generate feature responses, while the gated convolution branch uses 3×3 convolutions and sigmoid activation to generate gate weights. The output of the gated convolutional coding unit is: ； in, Indicates input features, Indicates output features, Represents the feature convolution branch, Indicates a gated convolution branch. Represents a non-linear activation function. This represents the Sigmoid activation function. This indicates element-wise multiplication; The bottleneck layer of the shared encoder incorporates a multi-receptive-field feature fusion module. This module consists of three parallel dilated convolutional branches with dilation rates of 1, 2, and 3, respectively, used to extract local structural features, mesoscale structural features, and larger receptive-field contextual features. The outputs of each dilated convolutional branch are concatenated through channels, then fused using either a 1×1 or 3×3 convolution. These channels are then residually connected to the bottleneck layer input features to obtain the bottleneck fused feature, expressed as: ； in, This represents the input features of the bottleneck layer. , , These represent dilated convolution branches with dilation rates of 1, 2, and 3, respectively. Indicates fused convolution, Indicates bottleneck fusion characteristics; The final decoding stage of the reconstruction decoder includes a structure feedback-gated fusion layer, which concatenates the final reconstruction decoding features, the backbone structure prediction map, and the detail structure prediction map through channel stitching. This concatenation is then performed using gated convolution to generate the final restored image. The expression for the structure feedback-gated fusion is as follows: ； in, This represents the final stage decoding features of the reconstruction decoder. This represents a prediction diagram of the main structure. This represents a detailed structure prediction plot. This represents the convolutional mapping used to generate structural feedback candidate features. This indicates element-wise multiplication. Indicates the structural feedback gating weights. This indicates the structural feedback fusion feature.

5. The method for repairing artifacts in a two-dimensional pseudocolor retinal nerve fiber layer thickness map according to claim 1, characterized in that, The artifact restoration network is trained using a phased training strategy, which includes: In the first stage, the reconstruction decoder and trunk structure decoder of the two-dimensional pseudo-color retinal nerve fiber layer thickness map are trained so that the reconstruction decoder outputs the coarse repair image I_base and the trunk structure decoder outputs the trunk structure prediction map. In the second stage, the reconstruction decoder, backbone structure decoder, and detail structure decoder are jointly trained, and the structural feedback information is made available to be backpropagated through the gradient of the reconstruction decoder, so that the backbone structure prediction map, detail structure prediction map, and final repaired image are jointly optimized. In the third stage, based on the joint training in the second stage, multi-layer diagnostic feature consistency loss is introduced for diagnostic-friendly fine-tuning, so that the repaired two-dimensional pseudo-color retinal nerve fiber layer thickness map can maintain glaucoma diagnostic features while restoring visual structure. The three-stage training strategy proceeds in the following order: prioritizing basic repair and backbone structure, followed by detailed structure learning, joint optimization of all branches, and fine-tuning with diagnostic features. The artifact repair network is trained using the Adam optimizer and employs the ReduceLROnPlateau learning rate scheduling strategy, with an initial learning rate of 0.0025. The learning rate is automatically reduced when the validation loss plateaus.

6. The method for repairing artifacts in a two-dimensional pseudocolor retinal nerve fiber layer thickness map according to claim 1, characterized in that, In the joint loss described in step S6, The total loss function is a weighted sum, and the specific formula is as follows: ； in , , , These are the weight hyperparameters for the basic reconstruction loss, structural coupling loss, boundary ring gradient consistency loss, and multi-layer diagnostic feature consistency loss, respectively. The basic reconstruction loss is used to constrain the pixel differences between the restored image and the reference artifact-free image, and its calculation formula is as follows: ； in, This image shows the thickness of the nerve fiber layer in the repaired two-dimensional pseudocolor retina. Indicates a reference image without artifacts. Indicates the artifact region mask. Indicates the valid region mask. Indicates the effective region reconstruction weight. Indicates the reconstruction weight for invalid regions; The structural coupling loss is used to ensure that the trunk structure representation and detail structure representation obtained in the final restored image are consistent with the trunk structure prediction map and detail structure prediction map, respectively. Its calculation formula is as follows: ； ； in, This indicates the restoration of the image's main structure. Represents the main structure of a real image. This indicates the restoration of the image's detailed structure. Represents the detailed structure of a real image; Multilayer diagnostic feature consistency loss is used to constrain the preservation of glaucoma diagnostic-related discriminative information in the restored image. Its calculation formula is as follows: ； in, The diagnostic feature extraction network is represented in the first... The feature representation of each preset level output. Indicates the first Weights at each level.

Images