Non-missing image intelligent restoration method based on self-supervised learning

Abstract

The invention discloses a non-abandoned image intelligent restoration method based on self-supervised learning, and the method comprises the steps: S1, collecting an image sequence, segmenting a semantic region, synthesizing a damage mask, and generating an original sample set; s2, through a space-time decoupling encoder, utilizing a ResNeSt network to extract high-frequency textures, capturing motion contexts, and obtaining an initial feature map; s3, constructing an improved N-BEATS model to decompose a time sequence component, and performing bidirectional prediction to output a periodic feature map; s4, calculating a confidence coefficient weight fusion feature, and performing dynamic smoothing and calibration by using a bidirectional difference consistency constraint factor to generate a to-be-repaired feature; s5, constructing a joint self-supervised loss function to predict mask pixels and enhance feature consistency, and outputting an iterative optimization target; and S6, performing back propagation to update the model and minimize loss, and outputting a final high-definition image. According to the method, the problems of texture blurring and time sequence jitter in non-residual image restoration are solved, and the restoration quality and continuity are improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to an intelligent restoration method for intangible cultural heritage images based on self-supervised learning. Background Technology

[0002] Deep learning models, with their feature extraction and trend prediction capabilities in nonlinear spatiotemporal sequence modeling, have been widely applied in recent years in fields such as film and television restoration, video enhancement, and cultural heritage protection, becoming an important development direction for intelligent image restoration. However, in practical applications, intangible cultural heritage image restoration scenarios face many challenges, including complex damage types, severe spatiotemporal degradation coupling, and difficulty in preserving inter-frame flickering details. The deployment effectiveness of existing deep restoration models is still constrained by many factors.

[0003] Most current image restoration methods rely on a single spatial domain feature input, making it difficult to fully utilize complementary information such as single-frame texture details and multi-frame temporal dynamic context, resulting in a lack of temporal coherence in the reconstruction of damaged areas. Some systems only use a single pixel-level loss function to evaluate the model output, ignoring the comprehensive influence of multiple dimensions such as denoising effect, scratch removal integrity, color restoration accuracy, and flicker elimination stability, thus limiting the adaptive optimization capability of restoration strategies. Furthermore, the restoration process lacks visualization and consistency constraints on intermediate features, making it difficult to provide clear intermediate feedback to restoration personnel, affecting the reliability and usability of the restoration results.

[0004] Furthermore, most of the loss constraints of existing deep learning models in video restoration are statically designed, failing to dynamically adjust the fusion weights of each feature component based on the confidence level of the damaged area and the prediction error. This results in some areas with blurred textures or violent motion performing poorly for a long time without being able to effectively correct the errors. It is difficult to adapt to changes in intangible cultural heritage video scenes with different degrees of damage and types of motion, seriously affecting the practical value and stability of the model in the digitization of real cultural heritage.

[0005] Therefore, how to provide a method for intelligent restoration of intangible cultural heritage images based on self-supervised learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an intelligent restoration method for intangible cultural heritage images based on self-supervised learning. This invention fully integrates key steps such as ResNeSt network spatial texture extraction, improved N-BEATS model bidirectional temporal prediction, gated spatiotemporal fusion, and bidirectional differential consistency constraint factor calibration, constructing an intelligent image restoration process with high-frequency detail capture, bidirectional temporal trend backtracking, adaptive feature weighting, and physical consistency calibration. This invention utilizes the ResNeSt network to enhance the expression of spatial texture features, and achieves precise correction and dynamic smoothing of temporal evolution through the improved N-BEATS model combined with bidirectional differential consistency constraint factors. It possesses advantages such as bidirectional complementary temporal modeling, high preservation of damage restoration details, strong adaptability of fusion weight adjustment, and significant flicker elimination effect. It can significantly improve the clarity, color fidelity, and temporal stability of reconstructed intangible cultural heritage images under complex damage, thereby effectively solving problems such as texture detail loss, insufficient utilization of long-distance motion context, and dynamic image flicker in existing methods.

[0007] According to an embodiment of the present invention, a method for intelligent restoration of intangible cultural heritage images based on self-supervised learning includes the following steps:

[0008] S1. Synchronously acquire the intangible cultural heritage image sequence to be restored, use a pre-trained image segmentation algorithm to divide the semantic region of a single frame of intangible cultural heritage image, synthesize a simulated damage mask, and generate the original sample set.

[0009] S2. Input the original sample set into the pre-built spatiotemporal decoupled encoder, extract the high-frequency texture details of the single frame of intangible cultural heritage image through the ResNeSt network in the spatial flow, capture the long-distance motion context between consecutive frames, and obtain the initial feature map.

[0010] S3. Construct an improved N-BEATS model to decompose the numerical changes of intangible cultural heritage image pixels along the time axis into trend components and seasonal components. Use the improved N-BEATS model to perform forward prediction and backward backtracking along the time axis and output a periodic feature map.

[0011] S4. Calculate the confidence weight of the damaged region in the improved N-BEATS model, and perform adaptive weighted fusion of the initial feature map and the periodic feature map. Calculate the difference between the forward prediction and the backward backtracking to obtain the bidirectional difference consistency constraint factor, and perform dynamic smoothing and physical consistency calibration on the fused features to generate time-stable features to be repaired.

[0012] S5. Construct a joint self-supervised loss function for joint mask image modeling and global contrastive learning. Use the mask image modeling task to predict the pixel values ​​of the masked area. Enhance the feature consistency of the same intangible cultural heritage image under different enhanced views through cross-view contrastive learning constraints, and output the iterative optimization target.

[0013] S6. Calculate the gradient error based on the iterative optimization objective, perform backpropagation updates on the spatiotemporal decoupled encoder and the improved N-BEATS model and minimize the joint loss function, use the overall restoration architecture after training convergence to process the data to be processed, and output the final high-definition intangible cultural heritage images.

[0014] Optionally, S1 specifically includes: reading the intangible cultural heritage image sequence to be repaired from the storage device, calling a pre-trained image segmentation algorithm to process each frame of intangible cultural heritage image, dividing the pixels in the intangible cultural heritage image into different semantic regions, obtaining the semantic category and texture characteristics of each semantic region, calculating the corresponding damage probability based on the semantic category and texture characteristics, generating a simulated damage mask based on the damage probability, covering the simulated damage mask onto the corresponding semantic region, and generating an original sample set.

[0015] Optionally, S2 specifically includes:

[0016] S21. Input the original sample set into the pre-constructed spatiotemporal decoupling encoder, read the pixel data of a single frame of intangible cultural heritage image from the original sample set, input the pixel data of the single frame of intangible cultural heritage image into the ResNeSt network in the spatial stream, divide the input feature channels evenly into a preset number of feature groups in the ResNeSt network, and further divide the channels in each feature group into a preset number of bases.

[0017] S22. Perform convolution operations on the feature maps within each cardinality, with a kernel size of 1x1 and two other kernels that are increased according to a preset ratio. Then, sum the output feature maps element by element. Calculate the attention weights for each cardinality using a preset fully connected layer. Multiply the summed feature maps with the corresponding cardinality attention weights. Then, stitch the weighted feature maps of each cardinality along the channel dimension to extract the high-frequency texture details of a single frame of intangible cultural heritage image and output the spatial feature map.

[0018] S23. Input the continuous frame intangible cultural heritage image sequence from the original sample set into the SwinTransformer in the time stream, extract the position information and pixel change information between consecutive frames from the continuous frame intangible cultural heritage image sequence, capture the long-distance motion context between continuous frames, and output the time feature map.

[0019] S24. Read the number of channels and size of the spatial feature map and the temporal feature map, concatenate the spatial feature map and the temporal feature map in the channel dimension, calculate the sum of all pixel values ​​in the concatenated feature matrix, divide the sum by the total number of pixels to obtain the feature mean, and calculate the square of the difference between each pixel value and the feature mean.

[0020] S25. Sum all the squared differences and divide by the total number of pixels to get the variance. Calculate the square root of the variance to get the standard deviation. Use the feature mean and standard deviation to normalize the spliced ​​feature matrix to get the initial feature map.

[0021] Optionally, S3 specifically includes:

[0022] S31. Construct an improved N-BEATS model. Read the time series values ​​of each pixel in the initial feature map and input them into the forward fully connected layer and the backward fully connected layer of the improved N-BEATS model. Calculate the product of the time series values ​​and the preset weight matrix of the forward fully connected layer and add a preset positive bias term. Calculate the product of the time series values ​​and the preset weight matrix of the backward fully connected layer and add a preset negative bias term. Process the data through the ReLU activation function to obtain the positive trend features and the negative trend features.

[0023] S32. Input the positive trend features into the trend branch, use a 1x1 convolution kernel to perform sliding convolution calculation along the time axis, extract the low-frequency polynomial coefficients of pixel values ​​changing with time, and use them as trend components. Input the positive trend features into the seasonal branch, use Fourier transform to map the time domain values ​​to the frequency domain, extract the harmonic coefficients in the frequency domain, and project the harmonic coefficients back to the time domain to obtain the seasonal components.

[0024] S33. Set up a residual connection layer inside the improved N-BEATS model, read the original time series values, subtract the trend component from the original time series values ​​to obtain the detrended intermediate series values, and subtract the seasonal component to obtain the residual series values. Input the residual series values ​​into the next stacking layer for the second decomposition, and repeat the stacking decomposition for a preset number of times until the variance of the residual series values ​​is less than the preset variance threshold.

[0025] S34. Using the improved N-BEATS model, perform forward prediction along the time axis, extract the pixel values ​​within the preset length historical window before the current time, calculate the product of the pixel values ​​within the historical window and the preset weight matrix through a fully connected layer, and add a preset bias term. Use the ReLU activation function to process and obtain the prediction result, and use the prediction result as the pixel value prediction vector for the future time and save it as the forward prediction parameter.

[0026] S35. Backtrack along the time axis, extract the pixel values ​​within a future window of a preset length after the current time, calculate the product of the pixel values ​​within the future window and the transpose of the preset weight matrix through a fully connected layer, and add a preset bias term. Process the backtracking result using the ReLU activation function, and use the backtracking result as the pixel value backtracking vector of the historical time and save it as the backward backtracking parameter.

[0027] S36. Read the forward prediction parameters and the backward backtracking parameters, calculate the absolute value of the difference between each value in the forward prediction parameters and the corresponding value in the backward backtracking parameters, multiply the absolute value of the difference by the preset fusion coefficient to obtain the correction weight, use the correction weight to adjust the forward prediction parameters, map the adjusted forward prediction parameters to the channel dimension of the feature map, and generate a periodic feature map.

[0028] Optionally, S4 specifically includes:

[0029] S41. In the improved N-BEATS model, the pixel values ​​at corresponding positions in the initial feature map and the periodic feature map are read, and the pixel values ​​of the initial feature map and the periodic feature map are input into the Sigmoid activation function layer in the gated spatiotemporal fusion network to calculate the confidence weights with values ​​between 0 and 1.

[0030] S42. Multiply the confidence weights element-wise with the initial feature map and the periodic feature map respectively to obtain the weighted initial feature map and the weighted periodic feature map. Add the weighted initial feature map and the weighted periodic feature map to output the fused feature map after adaptive weighted fusion.

[0031] S43. Read the forward prediction parameters and the backward backtracking parameters. Subtract the value of the corresponding time from the value of the backward backtracking parameters at each time step in the forward prediction parameters to calculate the difference. Take the absolute value of the difference to obtain the absolute value of the two-way difference. Add all the absolute values ​​of the two-way difference and divide by the total number of time steps to obtain the two-way difference consistency constraint factor.

[0032] S44. Read the fused feature map and the bidirectional difference consistency constraint factor, dynamically adjust the standard deviation of the Gaussian filter kernel using the 3x3 Gaussian filter kernel matrix and the bidirectional difference consistency constraint factor, dynamically smooth the fused feature map, and output the dynamically smoothed feature map.

[0033] S45. Calculate the inter-frame difference between the current frame pixel value and the previous frame pixel value in the dynamic smoothing feature map. When the inter-frame difference exceeds the preset physical threshold, replace the current frame pixel value with the previous frame pixel value to perform physical consistency calibration on the dynamic smoothing feature map and generate a time-stable feature to be repaired.

[0034] Optionally, S44 specifically includes:

[0035] S441. Read the fused feature map and the bidirectional difference consistency constraint factor, input the value of the bidirectional difference consistency constraint factor into the Gaussian kernel generation function, and calculate the dynamically adjusted Gaussian distribution standard deviation by multiplying the value with the preset basic standard deviation coefficient.

[0036] S442. Generate a Gaussian filter weight matrix based on the dynamically adjusted standard deviation of the Gaussian distribution. Determine the size of the filter window as a 3x3 pixel area. Substitute each coordinate position within the 3x3 pixel area into the Gaussian probability density formula to calculate the weight value. Combine the calculated dynamically adjusted standard deviation of the Gaussian distribution to calculate the Gaussian weight value corresponding to each coordinate point and arrange them according to the window coordinate position to construct a 3x3 Gaussian filter kernel matrix containing different weight values.

[0037] S443. Using a 3x3 Gaussian filter kernel matrix, a sliding window operation is performed on each pixel of the fused feature map. The pixel gray value of each neighboring pixel in the window is read, and the pixel gray value of each neighboring pixel is multiplied by the corresponding Gaussian filter kernel weight value to calculate the product result.

[0038] S444. Sum the product results of all positions within the window, divide the sum by the sum of all weights in the 3x3 Gaussian filter kernel matrix to obtain the filtered gray value of the center pixel, replace all pixels in the fused feature map with the corresponding filtered gray value, and output the dynamically smoothed feature map.

[0039] Optionally, S5 specifically includes:

[0040] S51. Read the intangible cultural heritage images in the original sample set, input a single frame of intangible cultural heritage image into the mask generator, randomly select the pixel position according to the preset mask ratio, set the pixel value of the selected pixel to zero, generate a mask image, retain the unmasked pixel area as the visible area, input the pixel value in the visible area into the spatiotemporal decoupling encoder, extract the context feature vector of the visible area and calculate the predicted pixel value of the masked area.

[0041] S52. Subtract the predicted pixel value from the actual pixel value at the corresponding position in the original intangible cultural heritage image, calculate the square of the pixel difference, sum the squared differences of all mask positions and divide by the total number of mask pixels to obtain the mask image modeling loss value.

[0042] S53. Construct two data augmentation views. Perform random rotation, random cropping, and color dithering operations on the same frame of intangible cultural heritage image to generate a first augmentation view and a second augmentation view. Input the first augmentation view into a feature extraction network containing three convolutional layers. Use a preset convolution kernel to perform sliding window convolution calculation on the first augmentation view to extract local texture features. After downsampling, obtain the feature vector of the first view.

[0043] S54. Input the second enhanced view into a feature extraction network containing three convolutional layers, perform sliding window convolution calculation on the second enhanced view using a preset convolutional kernel, extract local texture features, obtain the second view feature vector after downsampling, calculate the Euclidean distance between the first view feature vector and the second view feature vector, perform negative exponential mapping on the Euclidean distance using an exponential function, and use the mapping result as the cross-view contrastive learning loss value.

[0044] S55. Read the mask image modeling loss value and the cross-view comparison learning loss value, multiply the mask image modeling loss value by the preset first weight coefficient, multiply the cross-view comparison learning loss value by the preset second weight coefficient, add the two product results, construct the joint self-supervised loss function, and output the iterative optimization objective.

[0045] Optionally, S6 specifically includes:

[0046] S61. Read the value of the joint self-supervised loss function in the iterative optimization objective, calculate the partial derivative of the value of the joint self-supervised loss function with respect to the weight matrix of the ResNeSt network in the spatiotemporal decoupled encoder to obtain the spatial flow gradient error, and at the same time calculate the partial derivative with respect to the weight matrix of the SwinTransformer to obtain the temporal flow gradient error, and the partial derivative with respect to the weight matrix of the fully connected layer in the improved N-BEATS model to obtain the temporal prediction gradient error.

[0047] S62. Multiply the spatial flow gradient error, temporal flow gradient error, and temporal prediction gradient error by the preset learning rate parameter to obtain the first weight update amount of the ResNeSt network, the second weight update amount of the SwinTransformer, and the third weight update amount of the improved N-BEATS model. Subtract the corresponding weight update amount from the current weight values ​​of the ResNeSt network, SwinTransformer, and improved N-BEATS model to complete one backpropagation update.

[0048] S63. Calculate the absolute value of the difference between the current joint self-supervised loss function value and the joint self-supervised loss function value of the previous iteration. When the absolute value of the difference is less than the preset convergence threshold, it is determined that the training has converged. Lock the internal parameters of the spatiotemporal decoupled encoder and the improved N-BEATS model to construct the overall repair architecture.

[0049] S64. Read the data to be processed, input the data to be processed into the overall repair architecture, use the spatiotemporal decoupling encoder to extract the spatial texture and temporal dynamic features of the data to be processed, and output the repaired pixel values ​​through the improved N-BEATS model.

[0050] S65. Replace the pixel values ​​in the grayscale abrupt change areas of the repaired pixel values ​​with the average value of the neighboring pixels, map the pixel color values ​​to the standard color gamut range, limit the grayscale fluctuations between consecutive frames to a preset range, and output the final high-definition intangible cultural heritage image.

[0051] The beneficial effects of this invention are:

[0052] First, by collecting the intangible cultural heritage image sequence to be restored and using image segmentation algorithms to divide semantic regions, an original sample set containing specific semantic degradation information was constructed, providing the model with a targeted distribution of damage types and effectively improving the model's ability to perceive texture damage in different semantic regions.

[0053] Secondly, a spatiotemporal decoupled encoder is used to decouple the extraction of spatial texture and temporal dynamic features. The ResNeSt network is used to introduce feature grouping and multi-scale convolutional attention mechanism, which enhances the ability to capture high-frequency texture details and solves the problem of texture blurring in single-frame restoration. At the same time, the improved N-BEATS model is combined to perform bidirectional temporal prediction and residual stacking decomposition, and the bidirectional differential consistency constraint factor is used to dynamically calibrate the forward prediction and backward backtracking, which significantly improves the modeling accuracy of long-distance motion context and effectively suppresses temporal jitter and screen flicker during the restoration process.

[0054] Furthermore, by employing a self-supervised loss function that combines masked image modeling with cross-view contrastive learning, the model is forced to learn pixel reconstruction of the masked region and enhance feature consistency across views, thus improving feature robustness, even without manual labeling. During training, the gradient error is calculated using the joint loss function, and backpropagation updates are performed on the spatiotemporally decoupled encoder and the improved N-BEATS model. By minimizing the joint loss, collaborative optimization of each network module is achieved, significantly improving the model's generalization ability and adaptive repair level in complex damage scenarios.

[0055] In summary, this invention significantly improves the denoising effect, scratch removal integrity, color reproduction accuracy, and temporal stability of intangible cultural heritage images by integrating ResNeSt network spatial feature enhancement, improved N-BEATS model bidirectional temporal modeling, and bidirectional differential consistency constraint factor dynamic calibration, thus achieving high-quality intelligent image restoration. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0057] Figure 1 This is a flowchart of an intelligent restoration method for intangible cultural heritage images based on self-supervised learning proposed in this invention;

[0058] Figure 2 This is a flowchart of the high-frequency texture detail extraction and initial feature map generation based on the ResNeSt network proposed in this invention.

[0059] Figure 3 This is a flowchart of the bidirectional temporal decomposition and periodic feature map output based on the improved N-BEATS model proposed in this invention. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0061] refer to Figures 1-3 A method for intelligent restoration of intangible cultural heritage images based on self-supervised learning includes the following steps:

[0062] S1. Synchronously acquire the intangible cultural heritage image sequence to be restored, use a pre-trained image segmentation algorithm to divide the semantic region of a single frame of intangible cultural heritage image, and synthesize a simulated damage mask that matches the semantic category and texture characteristics of each region to generate an original sample set containing specific semantic degradation information.

[0063] S2. Input the original sample set into the pre-built spatiotemporal decoupled encoder, extract the high-frequency texture details of the single frame of intangible cultural heritage image through the ResNeSt network in the spatial flow, and use the SwinTransformer in the temporal flow to capture the long-distance motion context between consecutive frames, so as to obtain an initial feature map that integrates spatial texture and temporal dynamics.

[0064] S3. Construct an improved N-BEATS model to decompose the numerical changes of intangible cultural heritage image pixels along the time axis into trend components and seasonal components. Use the improved N-BEATS model to perform forward prediction and backward backtracking along the time axis, and output a periodic feature map containing forward prediction parameters and backward backtracking parameters.

[0065] S4. In the improved N-BEATS model, the confidence weight of the damaged region is calculated by gated spatiotemporal fusion, and the initial feature map and periodic feature map are adaptively weighted and fused. The difference between the forward prediction and the backward backtracking is calculated to obtain the bidirectional difference consistency constraint factor. The fused features are dynamically smoothed and physically consistent calibrated to generate time-stable features to be repaired.

[0066] S5. Construct a self-supervised loss function that combines masked image modeling and global contrastive learning. Use the masked image modeling task to force the model to predict the pixel values ​​of the masked areas. Enhance the feature consistency of the same intangible cultural heritage image under different enhanced views through cross-view contrastive learning constraints, and output the iterative optimization target.

[0067] S6. Calculate the gradient error based on the iterative optimization objective, perform backpropagation updates on the spatiotemporal decoupled encoder and the improved N-BEATS model, minimize the joint loss function, use the overall restoration architecture after training convergence to process the data to be processed, and output the final high-definition intangible cultural heritage image after denoising, scratch removal, color restoration and flicker elimination.

[0068] This invention significantly improves the restoration quality and visual coherence of intangible cultural heritage (ICH) images. Through a spatiotemporally decoupled encoder, a ResNeSt network is used to accurately extract high-frequency texture details from single frames, combined with a SwingTransformer to capture long-distance motion context, enabling in-depth analysis of complex image content. An improved N-BEATS model is used for bidirectional temporal decomposition and prediction of pixel changes, combined with bidirectional differential consistency constraints, effectively solving the problems of flickering and temporal jitter in dynamic images. A self-supervised mechanism of joint mask image modeling and global contrastive learning is introduced to enhance feature consistency and significantly improve the accuracy of denoising, scratch removal, and color restoration. This method fully leverages the synergistic advantages of the ResNeSt network and the improved N-BEATS model, ensuring temporal stability while outputting high-definition ICH images with rich details and clear image quality, providing efficient and reliable technical support for the digital preservation of cultural heritage.

[0069] In this embodiment, S1 specifically includes: reading the intangible cultural heritage image sequence to be restored from the storage device, calling a pre-trained image segmentation algorithm to process each frame of intangible cultural heritage image, dividing the pixels in the intangible cultural heritage image into different semantic regions, obtaining the semantic category and texture characteristics of each semantic region, calculating the corresponding damage probability based on the semantic category and texture characteristics, generating a simulated damage mask based on the damage probability, covering the simulated damage mask onto the corresponding semantic region, and generating an original sample set containing specific semantic degradation information.

[0070] For example, given a frame of intangible cultural heritage image containing a "human face" and a "background wall," the image segmentation algorithm divides it into two semantic regions. Because the "human face" has complex texture and contains key details of facial features, the model sets its damage probability to 0.1 (i.e., retaining 90% of pixels); while the "background wall" region has a simple texture and is prone to accumulating scratches, so its damage probability is set to 0.6 (i.e., retaining 40% of pixels). Based on these probabilities, the model randomly selects 60% of the pixels in the "background wall" region, sets the pixel values ​​of these points to 0 or adds Gaussian noise to generate a simulated damage mask, and overlays it onto the image, ultimately generating an original sample where the wall is severely damaged but the face remains intact.

[0071] In this embodiment, S2 specifically includes:

[0072] S21. Input the original sample set into the pre-constructed spatiotemporal decoupling encoder, read the pixel data of a single frame of intangible cultural heritage image from the original sample set, input the pixel data of the single frame of intangible cultural heritage image into the ResNeSt network in the spatial stream, divide the input feature channels evenly into a preset number of feature groups in the ResNeSt network, and further divide the channels in each feature group into a preset number of bases.

[0073] S22. Perform convolution operations on the feature maps within each cardinality, with a kernel size of 1x1 and two other kernels that are increased according to a preset ratio. Then, sum the output feature maps element by element. Calculate the attention weights for each cardinality using a preset fully connected layer. Multiply the summed feature maps with the corresponding cardinality attention weights. Then, stitch the weighted feature maps of each cardinality along the channel dimension to extract the high-frequency texture details of a single frame of intangible cultural heritage image and output the spatial feature map.

[0074] For example, the number of feature groups is set to 4, and the number of channels for each cardinality is 32. For feature maps within a certain cardinality, the model performs parallel convolution operations using 1x1, 3x3, and 5x5 convolution kernels, respectively, with each output feature map having a size of H×W×32. The three output feature maps are summed element-wise, and the attention weights for that cardinality are calculated using a fully connected layer as [0.2, 0.5, 0.3, 0.6]. The summed feature map is multiplied by the corresponding weights, and finally, the weighted feature maps of the four cardins are concatenated along the channel dimension to obtain a spatial feature map that integrates multi-scale features.

[0075] S23. Input the continuous frame intangible cultural heritage image sequence from the original sample set into the SwinTransformer in the time stream, extract the position information and pixel change information between consecutive frames from the continuous frame intangible cultural heritage image sequence, capture the long-distance motion context between continuous frames, and output the time feature map.

[0076] S24. Read the number of channels and size of the spatial feature map and the temporal feature map, concatenate the spatial feature map and the temporal feature map in the channel dimension, calculate the sum of all pixel values ​​in the concatenated feature matrix, divide the sum by the total number of pixels to obtain the feature mean, and calculate the square of the difference between each pixel value and the feature mean.

[0077] S25. Sum all the squared differences and divide by the total number of pixels to get the variance. Calculate the square root of the variance to get the standard deviation. Use the feature mean and standard deviation to normalize the spliced ​​feature matrix to get the initial feature map that integrates spatial texture and temporal dynamics.

[0078] This invention utilizes a spatiotemporal decoupled encoder combined with a ResNeSt network and a SwinTransformer to extract deep features from the spatial texture details and long-distance motion context of intangible cultural heritage images. The ResNeSt network performs cardinality-based multi-scale convolutions and attention weighting on the feature channels to accurately capture high-frequency texture details in a single frame. The SwinTransformer processes continuous frame sequences to extract position and pixel change information between consecutive frames, effectively capturing dynamic correlations. The spatial and temporal feature maps are concatenated and normalized based on mean and standard deviation to eliminate data distribution differences. This invention can fully integrate spatial texture and temporal dynamic information to generate initial feature maps with rich feature representation, significantly improving the model's ability to analyze complex image details and dynamic content, laying a solid foundation for subsequent high-precision image restoration.

[0079] In this embodiment, S3 specifically includes:

[0080] S31. Construct an improved N-BEATS model. Read the time series values ​​of each pixel in the initial feature map and input them into the forward fully connected layer and the backward fully connected layer of the improved N-BEATS model. Calculate the product of the time series values ​​and the preset weight matrix of the forward fully connected layer and add a preset positive bias term. Calculate the product of the time series values ​​and the preset weight matrix of the backward fully connected layer and add a preset negative bias term. Process the data through the ReLU activation function to obtain the positive trend features and the negative trend features.

[0081] S32. Input the positive trend features into the trend branch, use a 1x1 convolution kernel to perform sliding convolution calculation along the time axis, extract the low-frequency polynomial coefficients of pixel values ​​changing with time, and use them as trend components. Input the positive trend features into the seasonal branch, use Fourier transform to map the time domain values ​​to the frequency domain, extract the harmonic coefficients in the frequency domain, and project the harmonic coefficients back to the time domain to obtain the seasonal components.

[0082] S33. Set up a residual connection layer inside the improved N-BEATS model, read the original time series values, subtract the trend component from the original time series values ​​to obtain the detrended intermediate series values, and subtract the seasonal component to obtain the residual series values. Input the residual series values ​​into the next stacking layer for the second decomposition, and repeat the stacking decomposition for a preset number of times until the variance of the residual series values ​​is less than the preset variance threshold.

[0083] S34. Using the improved N-BEATS model, perform forward prediction along the time axis, extract the pixel values ​​within the preset length historical window before the current time, calculate the product of the pixel values ​​within the historical window and the preset weight matrix through a fully connected layer, and add a preset bias term. Use the ReLU activation function to process and obtain the prediction result, and use the prediction result as the pixel value prediction vector for the future time and save it as the forward prediction parameter.

[0084] For example, let the current time be t, and the preset historical window length be 5. The model extracts the pixel values ​​[120, 122, 125, 124, 123] from time t-5 to time t as input. The fully connected layer calculates the output prediction result 125 according to the preset weight matrix. This value is used as the pixel value prediction vector at time t+1, i.e., the forward prediction parameter.

[0085] S35. Backtrack along the time axis, extract the pixel values ​​within a future window of a preset length after the current time, calculate the product of the pixel values ​​within the future window and the transpose of the preset weight matrix through a fully connected layer, and add a preset bias term. Process the backtracking result using the ReLU activation function, and use the backtracking result as the pixel value backtracking vector of the historical time and save it as the backward backtracking parameter.

[0086] For example, let the current time be t, and the preset future window length be 5. The model extracts the pixel values ​​[125, 126, 128, 127, 126] from time t+1 to t+5 as input. Through the backtracking layer, the output backtracking result is 121, which serves as the backtracking estimate of the pixel value at time t-1, i.e., the backtracking parameter.

[0087] S36. Read the forward prediction parameters and the backward backtracking parameters, calculate the absolute value of the difference between each value in the forward prediction parameters and the corresponding value in the backward backtracking parameters, multiply the absolute value of the difference by the preset fusion coefficient to obtain the correction weight, use the correction weight to adjust the forward prediction parameters, map the adjusted forward prediction parameters to the channel dimension of the feature map, and generate a periodic feature map containing the forward prediction parameters and the backward backtracking parameters.

[0088] This invention achieves deep decomposition of pixel values ​​and extraction of periodic features in intangible cultural heritage images by introducing an improved N-BEATS model combined with a bidirectional temporal prediction mechanism. Fully connected layers and the ReLU activation function are used to process the time series bidirectionally, extracting forward and reverse trend features. Trend and seasonal components are obtained through 1x1 convolution and Fourier transform, respectively. Residual connected layers are set up for multi-layer stacking decomposition until the residual variance meets a threshold. This invention can perform forward prediction and backward backtracking along the time axis, using the absolute value of the difference to calculate correction weights for weighted adjustment of the feature map. This method effectively captures the low-frequency variation patterns and high-frequency harmonic information of the image sequence, significantly improving the accuracy of periodic feature expression and providing reliable data support for eliminating temporal jitter and generating stable restoration features.

[0089] In this embodiment, S4 specifically includes:

[0090] S41. In the improved N-BEATS model, the pixel values ​​at corresponding positions in the initial feature map and the periodic feature map are read, and the pixel values ​​of the initial feature map and the periodic feature map are input into the Sigmoid activation function layer in the gated spatiotemporal fusion network to calculate the confidence weights with values ​​between 0 and 1.

[0091] S42. Multiply the confidence weights element-wise with the initial feature map and the periodic feature map respectively to obtain the weighted initial feature map and the weighted periodic feature map. Add the weighted initial feature map and the weighted periodic feature map to output the fused feature map after adaptive weighted fusion.

[0092] S43. Read the forward prediction parameters and the backward backtracking parameters. Subtract the value of the corresponding time from the value of the backward backtracking parameters at each time step in the forward prediction parameters to calculate the difference. Take the absolute value of the difference to obtain the absolute value of the two-way difference. Add all the absolute values ​​of the two-way difference and divide by the total number of time steps to obtain the two-way difference consistency constraint factor.

[0093] S44. Read the fused feature map and the bidirectional difference consistency constraint factor, dynamically adjust the standard deviation of the Gaussian filter kernel using the 3x3 Gaussian filter kernel matrix and the bidirectional difference consistency constraint factor, dynamically smooth the fused feature map, and output the dynamically smoothed feature map.

[0094] S45. Calculate the inter-frame difference between the current frame pixel value and the previous frame pixel value in the dynamic smoothing feature map. When the inter-frame difference exceeds the preset physical threshold, replace the current frame pixel value with the previous frame pixel value to perform physical consistency calibration on the dynamic smoothing feature map and generate a time-stable feature to be repaired.

[0095] In this embodiment, S44 specifically includes:

[0096] S441. Read the fused feature map and the bidirectional difference consistency constraint factor, input the value of the bidirectional difference consistency constraint factor into the Gaussian kernel generation function, and calculate the dynamically adjusted Gaussian distribution standard deviation by multiplying the value with the preset basic standard deviation coefficient.

[0097] S442. Generate a Gaussian filter weight matrix based on the dynamically adjusted standard deviation of the Gaussian distribution. Determine the size of the filter window as a 3x3 pixel area. Substitute each coordinate position within the 3x3 pixel area into the Gaussian probability density formula to calculate the weight value. Combine the calculated dynamically adjusted standard deviation of the Gaussian distribution to calculate the Gaussian weight value corresponding to each coordinate point and arrange them according to the window coordinate position to construct a 3x3 Gaussian filter kernel matrix containing different weight values.

[0098] S443. Using a 3x3 Gaussian filter kernel matrix, a sliding window operation is performed on each pixel of the fused feature map. The pixel gray value of each neighboring pixel in the window is read, and the pixel gray value of each neighboring pixel is multiplied by the corresponding Gaussian filter kernel weight value to calculate the product result.

[0099] S444. Sum the product results of all positions within the window, divide the sum by the sum of all weights in the 3x3 Gaussian filter kernel matrix to obtain the filtered gray value of the center pixel, replace all pixels in the fused feature map with the corresponding filtered gray value, and output the dynamically smoothed feature map after dynamic smoothing.

[0100] This invention achieves dynamic optimization and enhanced temporal stability of intangible cultural heritage image restoration features by introducing gated spatiotemporal fusion and a two-way differential consistency constraint mechanism. The confidence weights of the initial feature map and the periodic feature map are calculated using the Sigmoid activation function to complete adaptive weighted fusion, balancing texture details and periodic motion information. A consistency constraint factor is constructed through differential operations on forward and backward prediction parameters, and dynamic Gaussian filtering is used to smooth the fused features, effectively suppressing high-frequency noise and flicker. When the inter-frame difference exceeds a physical threshold, the pixel values ​​of the previous frame are used for replacement calibration to ensure that image changes conform to physical laws. This invention can significantly eliminate temporal jitter and image anomalies during the restoration process, generating high-fidelity, coherent, and stable features to be restored, greatly improving the visual quality of dynamic images.

[0101] In this embodiment, S5 specifically includes:

[0102] S51. Read the intangible cultural heritage images in the original sample set, input a single frame of intangible cultural heritage image into the mask generator, randomly select the pixel position according to the preset mask ratio, set the pixel value of the selected pixel to zero, generate a mask image, retain the unmasked pixel area as the visible area, input the pixel value in the visible area into the spatiotemporal decoupling encoder, extract the context feature vector of the visible area and calculate the predicted pixel value of the masked area.

[0103] S52. Subtract the predicted pixel value from the actual pixel value at the corresponding position in the original intangible cultural heritage image, calculate the square of the pixel difference, sum the squared differences of all mask positions and divide by the total number of mask pixels to obtain the mask image modeling loss value.

[0104] S53. Construct two data augmentation views. Perform random rotation, random cropping, and color dithering operations on the same frame of intangible cultural heritage image to generate a first augmentation view and a second augmentation view. Input the first augmentation view into a feature extraction network containing three convolutional layers. Use a preset convolution kernel to perform sliding window convolution calculation on the first augmentation view to extract local texture features. After downsampling, obtain the feature vector of the first view.

[0105] S54. Input the second enhanced view into a feature extraction network containing three convolutional layers, perform sliding window convolution calculation on the second enhanced view using a preset convolutional kernel, extract local texture features, obtain the second view feature vector after downsampling, calculate the Euclidean distance between the first view feature vector and the second view feature vector, perform negative exponential mapping on the Euclidean distance using an exponential function, and use the mapping result as the cross-view contrastive learning loss value.

[0106] S55. Read the mask image modeling loss value and the cross-view comparison learning loss value, multiply the mask image modeling loss value by the preset first weight coefficient, multiply the cross-view comparison learning loss value by the preset second weight coefficient, add the two product results, construct the joint self-supervised loss function, and output the iterative optimization objective.

[0107] In this embodiment, S6 specifically includes:

[0108] S61. Read the value of the joint self-supervised loss function in the iterative optimization objective, calculate the partial derivative of the value of the joint self-supervised loss function with respect to the weight matrix of the ResNeSt network in the spatiotemporal decoupled encoder to obtain the spatial flow gradient error, and at the same time calculate the partial derivative with respect to the weight matrix of the SwinTransformer to obtain the temporal flow gradient error, and the partial derivative with respect to the weight matrix of the fully connected layer in the improved N-BEATS model to obtain the temporal prediction gradient error.

[0109] S62. Multiply the spatial flow gradient error, temporal flow gradient error, and temporal prediction gradient error by the preset learning rate parameter to obtain the first weight update amount of the ResNeSt network, the second weight update amount of the SwinTransformer, and the third weight update amount of the improved N-BEATS model. Subtract the corresponding weight update amount from the current weight values ​​of the ResNeSt network, SwinTransformer, and improved N-BEATS model to complete one backpropagation update.

[0110] S63. Calculate the absolute value of the difference between the current joint self-supervised loss function value and the joint self-supervised loss function value of the previous iteration. When the absolute value of the difference is less than the preset convergence threshold, it is determined that the training has converged. Lock the internal parameters of the spatiotemporal decoupled encoder and the improved N-BEATS model, and construct an overall repair architecture that includes the locked parameters.

[0111] S64. Read the data to be processed, input the data to be processed into the overall repair architecture, use the spatiotemporal decoupling encoder to extract the spatial texture and temporal dynamic features of the data to be processed, and output the repaired pixel values ​​through the improved N-BEATS model.

[0112] S65. Replace the pixel values ​​in the grayscale abrupt change areas of the repaired pixel values ​​with the average value of the neighboring pixels to achieve noise reduction and scratch removal. Map the pixel color values ​​to the standard color gamut range to achieve color restoration. Limit the grayscale fluctuations between consecutive frames to a preset range to achieve flicker elimination and output the final high-definition intangible cultural heritage image.

[0113] Example 1:

[0114] To verify the practical feasibility of this invention in the field of digital protection and intelligent restoration of intangible cultural heritage, it was deployed in the "Light and Shadow Memory" digital restoration project platform of a provincial intangible cultural heritage protection center. This platform is currently responsible for the digital rescue of film reels, old photographs, and early videotapes of high historical value throughout the province. The first batch of key restoration targets covers five major categories of intangible cultural heritage video materials, including shadow puppetry, local operas, and records of traditional handicrafts. These precious video materials, due to their age and harsh preservation conditions, generally suffer from severe image quality degradation, mainly manifested as mold adhesion, widespread scratches, severe color fading, noise accumulation, and frame instability caused by film shake. Furthermore, some of these materials are rare and unique copies, making restoration extremely difficult and requiring extremely high precision and safety. Traditional manual restoration methods mainly rely on software such as Photoshop for frame-by-frame processing, which is not only time-consuming and inefficient, but also difficult to guarantee the temporal consistency of consecutive frames, easily resulting in image flickering or distortion of moving objects, failing to meet the urgent need for large-scale intangible cultural heritage video rescue.

[0115] In practical deployment, the method of this invention first performs automated preprocessing and feature extraction on the original image sample set to be restored. The system inputs the collected single-frame intangible cultural heritage image pixel data into the ResNeSt network in the spatial stream. Utilizing its feature grouping and cardinality partitioning mechanism, combined with multi-scale convolutional attention, it accurately captures and enhances high-frequency texture details in the image, such as the embroidery patterns of costumes, facial micro-expressions, and the material texture of props, effectively removing surface blurring and smoothing effects. Simultaneously, the image sequence of consecutive frames is input into the SwinTransformer in the temporal stream. By extracting positional information and pixel change information between consecutive frames, it deeply captures long-distance motion contexts such as shadow puppet character movements and actor postures, generating stable temporal feature maps. Based on this, this invention utilizes a gated spatiotemporal fusion network to adaptively weight and fuse the extracted high-frequency texture features and temporal dynamic features, and combines an improved N-BEATS model for bidirectional temporal prediction and residual stacking decomposition. The introduced bidirectional differential consistency constraint factor dynamically calibrates the forward prediction and backward backtracking, significantly suppressing temporal jitter and image flicker during the repair process and ensuring the continuity of action sequences. Finally, the model is continuously optimized through a self-supervised loss function that combines mask image modeling and cross-view contrast learning, achieving unsupervised high-quality reconstruction of complex damage areas.

[0116] To objectively evaluate the restoration performance of this invention, the platform selected three representative typical intangible cultural heritage damage scenarios—"shadow puppet show dynamic images," "static restoration of old photos," and "local opera videotapes"—for a two-month comparative test. During the test, the method of this invention was rigorously compared with currently widely used traditional commercial restoration software (based on traditional filtering and convolutional neural network methods) under the same dataset and hardware environment. Key indicators examined included peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and restoration efficiency. Table 1 below shows the detailed performance comparison data between the method of this invention and traditional methods in different intangible cultural heritage image scenarios during the test period:

[0117] Table 1. Performance Comparison Data of the Invention and Traditional Methods in Intelligent Restoration of Intangible Cultural Heritage Images

[0118]

[0119] Based on the comparative data shown in Table 1, it can be seen that the intelligent restoration method proposed in this invention has significant performance advantages over traditional methods in the processing of intangible cultural heritage images, especially in terms of peak signal-to-noise ratio, structural similarity, restoration efficiency and temporal stability.

[0120] In terms of objective image quality, this invention achieves a PSNR of over 33dB and a stable SSIM of over 0.91 in three typical image scenarios, far exceeding the average level of approximately 28dB achieved by traditional methods. For example, in the "shadow puppet show dynamic image" scenario, traditional methods struggle to handle complex motion blur and texture details, achieving only a structural similarity of 0.812. In contrast, this invention accurately extracts high-frequency textures using a ResNeSt network and combines it with a spatiotemporal fusion mechanism, achieving an SSIM of 0.932 and 91% color fidelity, clearly restoring the fine textures and colors of the shadow puppets.

[0121] In terms of restoration timeliness, this invention relies on an optimized deep learning architecture to significantly reduce the average single-frame restoration time from more than 1 second in traditional methods to 0.12 to 0.21 seconds, improving restoration efficiency by nearly 5 to 6 times, thus meeting the needs for rapid rescue of massive image data.

[0122] Regarding temporal stability, addressing the jitter issue in dynamic images, this invention utilizes an improved N-BEATS model combined with bidirectional differential consistency calibration to successfully control the temporal jitter rate of "shadow puppetry" and "local operas" to within 2.3%, completely resolving the image flicker problem of up to 14% encountered by traditional methods. Overall, this invention maintains extremely high image quality while balancing processing speed and dynamic continuity, effectively filling the gaps in traditional technologies for the restoration of intangible cultural heritage dynamic images.

[0123] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent restoration of intangible cultural heritage images based on self-supervised learning, characterized in that, Includes the following steps: S1. Synchronously acquire the intangible cultural heritage image sequence to be restored, use a pre-trained image segmentation algorithm to divide the semantic region of a single frame of intangible cultural heritage image, synthesize a simulated damage mask, and generate the original sample set. S2. Input the original sample set into the pre-built spatiotemporal decoupled encoder, extract the high-frequency texture details of the single frame of intangible cultural heritage image through the ResNeSt network in the spatial flow, capture the long-distance motion context between consecutive frames, and obtain the initial feature map. S3. Construct an improved N-BEATS model to decompose the numerical changes of intangible cultural heritage image pixels along the time axis into trend components and seasonal components. Use the improved N-BEATS model to perform forward prediction and backward backtracking along the time axis and output a periodic feature map. S4. Calculate the confidence weight of the damaged region in the improved N-BEATS model, and perform adaptive weighted fusion of the initial feature map and the periodic feature map. Calculate the difference between the forward prediction and the backward backtracking to obtain the bidirectional difference consistency constraint factor, and perform dynamic smoothing and physical consistency calibration on the fused features to generate time-stable features to be repaired. S5. Construct a joint self-supervised loss function for joint mask image modeling and global contrastive learning. Use the mask image modeling task to predict the pixel values ​​of the masked area. Enhance the feature consistency of the same intangible cultural heritage image under different enhanced views through cross-view contrastive learning constraints, and output the iterative optimization target. S6. Calculate the gradient error based on the iterative optimization objective, perform backpropagation updates on the spatiotemporal decoupled encoder and the improved N-BEATS model and minimize the joint loss function, use the overall restoration architecture after training convergence to process the data to be processed, and output the final high-definition intangible cultural heritage images.

2. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S1 specifically includes: reading the intangible cultural heritage image sequence to be repaired from the storage device, calling a pre-trained image segmentation algorithm to process each frame of intangible cultural heritage image, dividing the pixels in the intangible cultural heritage image into different semantic regions, obtaining the semantic category and texture characteristics of each semantic region, calculating the corresponding damage probability based on the semantic category and texture characteristics, generating a simulated damage mask based on the damage probability, covering the simulated damage mask onto the corresponding semantic region, and generating the original sample set.

3. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S2 specifically includes: S21. Input the original sample set into the pre-constructed spatiotemporal decoupling encoder, read the pixel data of a single frame of intangible cultural heritage image from the original sample set, input the pixel data of the single frame of intangible cultural heritage image into the ResNeSt network in the spatial stream, divide the input feature channels evenly into a preset number of feature groups in the ResNeSt network, and further divide the channels in each feature group into a preset number of bases. S22. Perform convolution operations on the feature maps within each cardinality, with a kernel size of 1x1 and two other kernels that are increased according to a preset ratio. Then, sum the output feature maps element by element. Calculate the attention weights for each cardinality using a preset fully connected layer. Multiply the summed feature maps with the corresponding cardinality attention weights. Then, stitch the weighted feature maps of each cardinality along the channel dimension to extract the high-frequency texture details of a single frame of intangible cultural heritage image and output the spatial feature map. S23. Input the continuous frame intangible cultural heritage image sequence from the original sample set into the SwinTransformer in the time stream, extract the position information and pixel change information between consecutive frames from the continuous frame intangible cultural heritage image sequence, capture the long-distance motion context between continuous frames, and output the time feature map. S24. Read the number of channels and size of the spatial feature map and the temporal feature map, concatenate the spatial feature map and the temporal feature map in the channel dimension, calculate the sum of all pixel values ​​in the concatenated feature matrix, divide the sum by the total number of pixels to obtain the feature mean, and calculate the square of the difference between each pixel value and the feature mean. S25. Sum all the squared differences and divide by the total number of pixels to get the variance. Calculate the square root of the variance to get the standard deviation. Use the feature mean and standard deviation to normalize the spliced ​​feature matrix to get the initial feature map.

4. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S3 specifically includes: S31. Construct an improved N-BEATS model. Read the time series values ​​of each pixel in the initial feature map and input them into the forward fully connected layer and the backward fully connected layer of the improved N-BEATS model. Calculate the product of the time series values ​​and the preset weight matrix of the forward fully connected layer and add a preset positive bias term. Calculate the product of the time series values ​​and the preset weight matrix of the backward fully connected layer and add a preset negative bias term. Process the data through the ReLU activation function to obtain the positive trend features and the negative trend features. S32. Input the positive trend features into the trend branch, use a 1x1 convolution kernel to perform sliding convolution calculation along the time axis, extract the low-frequency polynomial coefficients of pixel values ​​changing with time, and use them as trend components. Input the positive trend features into the seasonal branch, use Fourier transform to map the time domain values ​​to the frequency domain, extract the harmonic coefficients in the frequency domain, and project the harmonic coefficients back to the time domain to obtain the seasonal components. S33. Set up a residual connection layer inside the improved N-BEATS model, read the original time series values, subtract the trend component from the original time series values ​​to obtain the detrended intermediate series values, and subtract the seasonal component to obtain the residual series values. Input the residual series values ​​into the next stacking layer for the second decomposition, and repeat the stacking decomposition for a preset number of times until the variance of the residual series values ​​is less than the preset variance threshold. S34. Using the improved N-BEATS model, perform forward prediction along the time axis, extract the pixel values ​​within the preset length historical window before the current time, calculate the product of the pixel values ​​within the historical window and the preset weight matrix through a fully connected layer, and add a preset bias term. Use the ReLU activation function to process and obtain the prediction result, and use the prediction result as the pixel value prediction vector for the future time and save it as the forward prediction parameter. S35. Backtrack along the time axis, extract the pixel values ​​within a future window of a preset length after the current time, calculate the product of the pixel values ​​within the future window and the transpose of the preset weight matrix through a fully connected layer, and add a preset bias term. Process the backtracking result using the ReLU activation function, and use the backtracking result as the pixel value backtracking vector of the historical time and save it as the backward backtracking parameter. S36. Read the forward prediction parameters and the backward backtracking parameters, calculate the absolute value of the difference between each value in the forward prediction parameters and the corresponding value in the backward backtracking parameters, multiply the absolute value of the difference by the preset fusion coefficient to obtain the correction weight, use the correction weight to adjust the forward prediction parameters, map the adjusted forward prediction parameters to the channel dimension of the feature map, and generate a periodic feature map.

5. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S4 specifically includes: S41. In the improved N-BEATS model, the pixel values ​​at corresponding positions in the initial feature map and the periodic feature map are read, and the pixel values ​​of the initial feature map and the periodic feature map are input into the Sigmoid activation function layer in the gated spatiotemporal fusion network to calculate the confidence weights with values ​​between 0 and 1. S42. Multiply the confidence weights element-wise with the initial feature map and the periodic feature map respectively to obtain the weighted initial feature map and the weighted periodic feature map. Add the weighted initial feature map and the weighted periodic feature map to output the fused feature map after adaptive weighted fusion. S43. Read the forward prediction parameters and the backward backtracking parameters. Subtract the value of the corresponding time from the value of the backward backtracking parameters at each time step in the forward prediction parameters to calculate the difference. Take the absolute value of the difference to obtain the absolute value of the two-way difference. Add all the absolute values ​​of the two-way difference and divide by the total number of time steps to obtain the two-way difference consistency constraint factor. S44. Read the fused feature map and the bidirectional difference consistency constraint factor, dynamically adjust the standard deviation of the Gaussian filter kernel using the 3x3 Gaussian filter kernel matrix and the bidirectional difference consistency constraint factor, dynamically smooth the fused feature map, and output the dynamically smoothed feature map. S45. Calculate the inter-frame difference between the current frame pixel value and the previous frame pixel value in the dynamic smoothing feature map. When the inter-frame difference exceeds the preset physical threshold, replace the current frame pixel value with the previous frame pixel value to perform physical consistency calibration on the dynamic smoothing feature map and generate a time-stable feature to be repaired.

6. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 5, characterized in that, S44 specifically includes: S441. Read the fused feature map and the bidirectional difference consistency constraint factor, input the value of the bidirectional difference consistency constraint factor into the Gaussian kernel generation function, and calculate the dynamically adjusted Gaussian distribution standard deviation by multiplying the value with the preset basic standard deviation coefficient. S442. Generate a Gaussian filter weight matrix based on the dynamically adjusted standard deviation of the Gaussian distribution. Determine the size of the filter window as a 3x3 pixel area. Substitute each coordinate position within the 3x3 pixel area into the Gaussian probability density formula to calculate the weight value. Combine the calculated dynamically adjusted standard deviation of the Gaussian distribution to calculate the Gaussian weight value corresponding to each coordinate point and arrange them according to the window coordinate position to construct a 3x3 Gaussian filter kernel matrix containing different weight values. S443. Using a 3x3 Gaussian filter kernel matrix, a sliding window operation is performed on each pixel of the fused feature map. The pixel gray value of each neighboring pixel in the window is read, and the pixel gray value of each neighboring pixel is multiplied by the corresponding Gaussian filter kernel weight value to calculate the product result. S444. Sum the product results of all positions within the window, divide the sum by the sum of all weights in the 3x3 Gaussian filter kernel matrix to obtain the filtered gray value of the center pixel, replace all pixels in the fused feature map with the corresponding filtered gray value, and output the dynamically smoothed feature map.

7. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S5 specifically includes: S51. Read the intangible cultural heritage images in the original sample set, input a single frame of intangible cultural heritage image into the mask generator, randomly select the pixel position according to the preset mask ratio, set the pixel value of the selected pixel to zero, generate a mask image, retain the unmasked pixel area as the visible area, input the pixel value in the visible area into the spatiotemporal decoupling encoder, extract the context feature vector of the visible area and calculate the predicted pixel value of the masked area. S52. Subtract the predicted pixel value from the actual pixel value at the corresponding position in the original intangible cultural heritage image, calculate the square of the pixel difference, sum the squared differences of all mask positions and divide by the total number of mask pixels to obtain the mask image modeling loss value. S53. Construct two data augmentation views. Perform random rotation, random cropping, and color dithering operations on the same frame of intangible cultural heritage image to generate a first augmentation view and a second augmentation view. Input the first augmentation view into a feature extraction network containing three convolutional layers. Use a preset convolution kernel to perform sliding window convolution calculation on the first augmentation view to extract local texture features. After downsampling, obtain the feature vector of the first view. S54. Input the second enhanced view into a feature extraction network containing three convolutional layers, perform sliding window convolution calculation on the second enhanced view using a preset convolutional kernel, extract local texture features, obtain the second view feature vector after downsampling, calculate the Euclidean distance between the first view feature vector and the second view feature vector, perform negative exponential mapping on the Euclidean distance using an exponential function, and use the mapping result as the cross-view contrastive learning loss value. S55. Read the mask image modeling loss value and the cross-view comparison learning loss value, multiply the mask image modeling loss value by the preset first weight coefficient, multiply the cross-view comparison learning loss value by the preset second weight coefficient, add the two product results, construct the joint self-supervised loss function, and output the iterative optimization objective.

8. The intelligent restoration method for intangible cultural heritage images based on self-supervised learning according to claim 1, characterized in that, S6 specifically includes: S61. Read the value of the joint self-supervised loss function in the iterative optimization objective, calculate the partial derivative of the value of the joint self-supervised loss function with respect to the weight matrix of the ResNeSt network in the spatiotemporal decoupled encoder to obtain the spatial flow gradient error, and at the same time calculate the partial derivative with respect to the weight matrix of the SwinTransformer to obtain the temporal flow gradient error, and the partial derivative with respect to the weight matrix of the fully connected layer in the improved N-BEATS model to obtain the temporal prediction gradient error. S62. Multiply the spatial flow gradient error, temporal flow gradient error, and temporal prediction gradient error by the preset learning rate parameter to obtain the first weight update amount of the ResNeSt network, the second weight update amount of the SwinTransformer, and the third weight update amount of the improved N-BEATS model. Subtract the corresponding weight update amount from the current weight values ​​of the ResNeSt network, SwinTransformer, and improved N-BEATS model to complete one backpropagation update. S63. Calculate the absolute value of the difference between the current joint self-supervised loss function value and the joint self-supervised loss function value of the previous iteration. When the absolute value of the difference is less than the preset convergence threshold, it is determined that the training has converged. Lock the internal parameters of the spatiotemporal decoupled encoder and the improved N-BEATS model to construct the overall repair architecture. S64. Read the data to be processed, input the data to be processed into the overall repair architecture, use the spatiotemporal decoupling encoder to extract the spatial texture and temporal dynamic features of the data to be processed, and output the repaired pixel values ​​through the improved N-BEATS model. S65. Replace the pixel values ​​in the grayscale abrupt change areas of the repaired pixel values ​​with the average value of the neighboring pixels, map the pixel color values ​​to the standard color gamut range, limit the grayscale fluctuations between consecutive frames to a preset range, and output the final high-definition intangible cultural heritage image.

Images