An image reconstruction method and related apparatus

Abstract

This invention discloses an image reconstruction method and related apparatus, relating to the field of image processing. The method involves acquiring an image of an ancient painting or calligraphy to be reconstructed, performing multi-scale degradation simulation on it, and then using the simulation results to adaptively correct degradation, obtaining a pre-processed image to be reconstructed. This pre-processed image is then subjected to blurry detail feature enhancement processing, and a target detail enhancement feature map is extracted from the enhanced pre-processed image. This map is then input into a pre-trained ancient painting or calligraphy image reconstruction model for image reconstruction, resulting in a reconstructed, clear image of the ancient painting or calligraphy. This invention effectively addresses non-uniform degradation by adaptively correcting the original image; it suppresses noise and enhances blurry details through blurry detail feature enhancement; and it introduces a quaternary loss function into the ancient painting or calligraphy image reconstruction model to maintain the original artwork's artistic spirit while improving image clarity.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image reconstruction method and related apparatus. Background Technology

[0002] In the field of cultural heritage digitization, the high-quality restoration of ancient paintings and calligraphy is facing severe challenges. After hundreds of years of natural aging, paintings and calligraphy generally exhibit complex degradation phenomena such as ink bleeding, fiber breakage, and pigment fading. These degradation phenomena are not only unevenly distributed in space, but also exhibit multi-scale characteristics in their manifestation.

[0003] Currently, there are two main image reconstruction methods. One method utilizes image enhancement techniques (such as histogram equalization) to reconstruct ancient calligraphy and painting images. However, image enhancement techniques employ a globally uniform processing approach, which struggles to adapt to the non-uniform degradation characteristics of calligraphy and painting in terms of space and scale. This often results in over-processing of detailed areas while under-processing of smooth areas. The other method is based on deep learning super-resolution methods. While this approach can improve the overall visual clarity of ancient calligraphy and painting images, it generally neglects the unique brushstroke textures and ink tones of calligraphy and painting, often resulting in artistic distortion in the reconstruction results—"clear but lacking spirit."

[0004] Therefore, how to provide an image reconstruction method that can effectively handle the non-uniform degradation of ancient calligraphy and painting images, enhance subtle artistic details, and maintain the original spirit has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention discloses an image reconstruction method and related apparatus, so as to effectively handle the non-uniform degradation phenomenon of ancient calligraphy and painting images, enhance subtle artistic details and maintain the original charm when reconstructing ancient calligraphy and painting images.

[0006] An image reconstruction method, comprising:

[0007] Obtain images of ancient paintings and calligraphy to be reconstructed;

[0008] Multi-scale degradation simulation is performed on the ancient calligraphy and painting image to be reconstructed, and adaptive degradation correction is performed on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results to obtain a preprocessed image to be reconstructed.

[0009] The preprocessed image to be reconstructed is subjected to blurry detail feature enhancement processing, and the target detail enhancement feature map is extracted from the enhanced preprocessed image to be reconstructed.

[0010] The target detail enhancement feature map is input into a pre-trained ancient calligraphy and painting image reconstruction model for calligraphy and painting image reconstruction processing to obtain a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met, with the minimization of the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

[0011] Optionally, multi-scale degradation simulation is performed on the ancient calligraphy and painting image to be reconstructed, and adaptive degradation correction is performed on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results to obtain a preprocessed image to be reconstructed, including:

[0012] For the ancient calligraphy and painting image to be reconstructed, at each spatial location and preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel;

[0013] The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales;

[0014] The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image to be reconstructed.

[0015] Optionally, the preprocessed image to be reconstructed is subjected to blurred detail feature enhancement processing, and the target detail enhancement feature map is extracted from the enhanced preprocessed image to be reconstructed, including:

[0016] For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude;

[0017] Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated.

[0018] Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region;

[0019] The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map.

[0020] Optionally, the training process of the ancient calligraphy and painting image reconstruction model includes:

[0021] A training dataset for reconstructing ancient calligraphy and painting images is constructed, wherein each sample in the training dataset is an ancient calligraphy and painting image pair, and the ancient calligraphy and painting image pair includes: an original ancient calligraphy and painting image with different degrees of degradation, and a high-quality clear reference image corresponding to the original ancient calligraphy and painting image;

[0022] Multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and the degradation simulation results are used to adaptively correct the degradation of the original ancient calligraphy and painting images to obtain preprocessed images;

[0023] The preprocessed image is subjected to blurry detail feature enhancement processing, and the detail enhancement feature map is extracted from the enhanced preprocessed image;

[0024] Based on the enhanced detail feature maps, the training dataset is trained using a supervised learning method to obtain the ancient calligraphy and painting image reconstruction model.

[0025] Optionally, multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and adaptive degradation correction is performed on the original ancient calligraphy and painting images using the degradation simulation results to obtain a preprocessed image, including:

[0026] For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel;

[0027] The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image.

[0028] Optionally, the preprocessed image is subjected to blurry detail feature enhancement processing, and a detail enhancement feature map is extracted from the enhanced preprocessed image, including:

[0029] For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude;

[0030] Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel;

[0031] Calculate the local enhancement factor based on the texture complexity and detail richness of the local region;

[0032] The multi-directional gradient magnitude, the channel fusion weight, and the local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate a detail-enhanced feature map.

[0033] Optionally, based on the detailed enhancement feature maps, the training dataset is trained using a supervised learning method to obtain the ancient calligraphy and painting image reconstruction model, including:

[0034] The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values;

[0035] A reconstructed, clear image of calligraphy and painting is generated based on the aforementioned detail enhancement values;

[0036] Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed.

[0037] For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is used for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

[0038] Optionally, the detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values, including:

[0039] A set of weights is calculated for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates.

[0040] The detail enhancement feature map is subjected to parallel convolution operations using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates;

[0041] The convolution results of the dilated convolution branches with different dilation rates are multiplied by their corresponding attention weights and summed. The summation result is then added to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0042] Optionally, generating a reconstructed clear calligraphy and painting image based on the detail enhancement value includes:

[0043] The detail enhancement map, composed of the aforementioned detail enhancement values, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstructed image.

[0044] Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and the reconstructed clear calligraphy and painting image is generated by combining a fully connected network.

[0045] An image reconstruction apparatus, comprising:

[0046] The image acquisition unit is used to acquire images of ancient paintings and calligraphy to be reconstructed.

[0047] The preprocessing unit is used to perform multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results, so as to obtain the preprocessed image to be reconstructed.

[0048] The feature enhancement unit is used to perform blurred detail feature enhancement processing on the preprocessed image to be reconstructed, and extract the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed;

[0049] The image reconstruction unit is used to input the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, thereby obtaining a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is trained by continuously optimizing the model parameters until the stopping iteration condition is met, with minimizing the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

[0050] A computer storage medium storing at least one instruction that, when executed by a processor, implements the image reconstruction method described above.

[0051] An electronic device, comprising: a memory and a processor;

[0052] The memory is used to store at least one instruction;

[0053] The processor is used to execute the at least one instruction to implement the image reconstruction method described above.

[0054] As can be seen from the above technical solution, the present invention discloses an image reconstruction method and related apparatus, which acquires an ancient calligraphy and painting image to be reconstructed, performs multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and uses the degradation simulation results to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed to obtain a preprocessed image to be reconstructed, performs blurry detail feature enhancement processing on the preprocessed image to be reconstructed, extracts the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed, and inputs the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model for calligraphy and painting image reconstruction processing to obtain a reconstructed clear ancient calligraphy and painting image. This invention simulates the degradation of original ancient calligraphy and painting images at multiple scales and uses the simulation results to adaptively correct the degradation of the images to be reconstructed. This effectively addresses the non-uniform degradation phenomena in space and scale, enhancing the detail and color information of the images. By enhancing the blurred detail features of the preprocessed images to be reconstructed, it effectively suppresses noise interference while selectively enhancing blurred details, enhancing subtle artistic details while preserving the overall structural information of the image. Furthermore, by introducing a quaternary loss function that integrates pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss during the training of the ancient calligraphy and painting image reconstruction model, it achieves the goal of improving image clarity while maintaining the original brushwork spirit and aesthetic characteristics. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the published drawings without creative effort.

[0056] Figure 1 This is a flowchart of an image reconstruction method disclosed in an embodiment of the present invention;

[0057] Figure 2 This is a flowchart of a training method for an ancient calligraphy and painting image reconstruction model disclosed in an embodiment of the present invention;

[0058] Figure 3 This is a bar chart comparing the quality of different methods at each feature extraction stage, as disclosed in an embodiment of the present invention.

[0059] Figure 4 This is a graph showing the peak signal-to-noise ratio as a function of scale number, as disclosed in an embodiment of the present invention.

[0060] Figure 5 This is a graph showing the change of structural similarity with scale number as disclosed in an embodiment of the present invention;

[0061] Figure 6 This is a graph showing the change in edge preservation degree as a function of scale number, as disclosed in an embodiment of the present invention.

[0062] Figure 7 This is a graph showing the change in processing time as a function of the number of scales, as disclosed in an embodiment of the present invention.

[0063] Figure 8 This is a schematic diagram of the structure of an image reconstruction device disclosed in an embodiment of the present invention;

[0064] Figure 9 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention discloses an image reconstruction method and related apparatus. By performing multi-scale degradation simulation on the original ancient calligraphy and painting image, and using the degradation simulation results to adaptively correct the degradation of the ancient calligraphy and painting image to be reconstructed, the non-uniform degradation phenomenon in space and scale of the image can be effectively handled, enhancing the detail and color information of the image. By performing fuzzy detail feature enhancement processing on the preprocessed image to be reconstructed, the fuzzy details can be enhanced in a targeted manner while effectively suppressing noise interference, enhancing subtle artistic details while preserving the overall structural information of the image. By introducing a quaternary loss function that integrates pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss during the training of the ancient calligraphy and painting image reconstruction model, the original brushwork spirit and aesthetic characteristics can be maintained while improving image clarity.

[0067] See Figure 1 The present invention discloses a flowchart of an image reconstruction method, which includes the following steps:

[0068] Step S101: Obtain the image of the ancient calligraphy and painting to be reconstructed.

[0069] Among them, the ancient calligraphy and painting images to be reconstructed refer to new, unprocessed ancient calligraphy and painting images.

[0070] Step S102: Perform multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and use the degradation simulation results to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed to obtain a preprocessed image to be reconstructed.

[0071] Ancient paintings and calligraphy images often exhibit degradation phenomena such as blurring, loss of detail, and color fading due to their age. This type of degradation is multi-scale and spatially non-uniform, meaning that the degree and pattern of degradation vary in different regions at different scales. Conventional techniques such as global normalization or histogram equalization, which assume that the degradation process is uniform across the entire image, are difficult to effectively handle this complex non-uniform degradation. This often leads to over-processing of detailed areas or under-processing of smooth areas, failing to provide high-quality preprocessing data for subsequent reconstruction stages.

[0072] This invention performs multi-scale degradation simulation on original ancient calligraphy and painting images, and uses the degradation simulation results to perform adaptive degradation correction on the ancient calligraphy and painting images to be reconstructed. This can effectively handle the non-uniform degradation phenomenon of images in space and scale, enhance the detail and color information of the images, and provide strong support for subsequent feature extraction work.

[0073] Step S103: Perform blur detail feature enhancement processing on the preprocessed image to be reconstructed, and extract the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed.

[0074] The blurred details in ancient calligraphy and painting images are often hidden in the low-frequency components or textures of the image. The edge and texture signals are relatively weak. When processing such images, conventional edge detection operators or high-frequency filters not only have difficulty in effectively distinguishing between real details and noise, but also tend to amplify the noise introduced during the image degradation process, causing the detailed features to be submerged and unable to provide effective feature representation for subsequent reconstruction.

[0075] Based on this, the present invention enhances the blurred details of the preprocessed image to be reconstructed by performing blurred detail feature enhancement processing. This can effectively suppress noise interference while enhancing blurred details in a targeted manner. While preserving the overall structural information of the image, it improves the recognizability of weak details, thus laying a solid foundation for the accurate reconstruction of ancient calligraphy and painting images. This replaces the edge detection or high-frequency filtering operations of conventional methods, thereby effectively solving the problem of destroying the original artistic brushstrokes in conventional methods.

[0076] Step S104: Input the target detail enhancement feature map into the pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, and obtain a reconstructed clear ancient calligraphy and painting image.

[0077] The ancient calligraphy and painting image reconstruction model is as follows: the training dataset for ancient calligraphy and painting image reconstruction is trained using a supervised learning method, with the minimization of the quaternary loss function as the training objective, and the model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met.

[0078] It should be noted that the quaternary loss function in this invention is a loss function that combines pixel fidelity loss, multi-scale perception loss, local stroke coherence loss, and rhythm consistency loss.

[0079] Among them, pixel fidelity loss can ensure that the reconstructed ancient calligraphy and painting images are as close as possible to the original images at the pixel level.

[0080] Multi-scale perceptual loss evaluates and optimizes images at different scales. At the macro scale, it ensures that the overall composition and layout of the reconstructed image are consistent with the original painting or calligraphy; at the micro scale, it focuses on the presentation of details, such as the thickness of brushstrokes and the variation in ink density. This multi-scale consideration ensures that the reconstructed image is highly similar to the original work at all levels, thus improving the overall image quality.

[0081] The brushstrokes in ancient Chinese calligraphy and painting are a crucial manifestation of their artistic style. Each stroke, from its beginning to its execution, embodies the artist's emotions and skill. The loss of continuity in local brushstrokes ensures the smoothness and consistency of the strokes during reconstruction, avoiding issues such as broken or disjointed strokes. For example, in reconstructing calligraphy, this ensures natural transitions and even pressure in the strokes, allowing the reconstructed work to retain its rhythm and flow.

[0082] The spirit and rhythm of calligraphy and painting are the soul of ancient Chinese art, embodying the overall artistic conception and appeal of the work. The quaternary loss function incorporates the loss of spirit and rhythm consistency, analyzing the spirit and rhythm characteristics of the original work, such as the fluidity of lines, the harmony of colors, and the appropriate density of composition, to preserve and inherit these spirit and rhythm as much as possible during the reconstruction process. This ensures that the reconstructed ancient calligraphy and painting not only resembles the original in appearance but also inherits its artistic essence, allowing viewers to experience the unique charm inherent in ancient calligraphy and painting.

[0083] Therefore, the quaternary loss function provides a comprehensive evaluation metric for training ancient calligraphy and painting reconstruction models. It comprehensively considers multiple aspects of the image, enabling the model to optimize from different angles during training and avoiding the local optima problem that may occur with a single loss function. By minimizing the quaternary loss function, the model can learn more comprehensive and accurate image features, thereby improving the accuracy and quality of reconstruction. It effectively overcomes the limitations of traditional U-Net and other networks with their single network structure and loss functions that only focus on pixel precision, achieving the collaborative reconstruction of artistic details and stylistic characteristics of calligraphy and painting images.

[0084] In summary, this invention discloses an image reconstruction method that involves acquiring an ancient painting or calligraphy image to be reconstructed, performing multi-scale degradation simulation on the image, using the degradation simulation results to perform adaptive degradation correction on the image to be reconstructed to obtain a preprocessed image to be reconstructed, performing blurry detail feature enhancement processing on the preprocessed image to be reconstructed, extracting a target detail enhancement feature map from the enhanced preprocessed image to be reconstructed, and inputting the target detail enhancement feature map into a pre-trained ancient painting or calligraphy image reconstruction model for painting or calligraphy image reconstruction processing to obtain a reconstructed clear ancient painting or calligraphy image. This invention simulates the degradation of original ancient calligraphy and painting images at multiple scales and uses the simulation results to adaptively correct the degradation of the images to be reconstructed. This effectively addresses the non-uniform degradation phenomena in space and scale, enhancing the detail and color information of the images. By enhancing the blurred detail features of the preprocessed images to be reconstructed, it effectively suppresses noise interference while selectively enhancing blurred details, enhancing subtle artistic details while preserving the overall structural information of the image. Furthermore, by introducing a quaternary loss function that integrates pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss during the training of the ancient calligraphy and painting image reconstruction model, it achieves the goal of improving image clarity while maintaining the original brushwork spirit and aesthetic characteristics.

[0085] In one embodiment, step S102 may specifically include:

[0086] (1) For the ancient calligraphy and painting image to be reconstructed, at each spatial location and at a preset scale, estimate the convolution kernel that represents the local blurring or degradation effect of the spatial location, and use it as the local degradation kernel;

[0087] The estimation of the local degradation kernel is achieved by minimizing the difference between a local image patch centered at the current location and corresponding scale, and the ideal sharp template. This ideal sharp template can be obtained from high-quality contemporaneous calligraphy and painting data, or it can be generated using prior knowledge of degradation.

[0088] (2) The degradation of the ancient calligraphy and painting image to be reconstructed is simulated at different scales using the local degradation kernel;

[0089] (3) The simulation results at each scale are adaptively fused according to the richness of details in the local area of ​​each location, and a bias term is added to adjust the color for adaptive degradation correction to obtain the preprocessed image to be reconstructed.

[0090] In one embodiment, step S103 may specifically include:

[0091] (1) For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude;

[0092] In practical applications, for each color channel of the preprocessed image, its gradient magnitude is calculated along four preset directions. This gradient magnitude represents the edge and texture change information of the preprocessed image in different directions.

[0093] (2) Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, calculate the target channel fusion weight corresponding to each color channel;

[0094] To adaptively fuse the gradient information of the three color channels, an adaptive weight is calculated for each spatial location, each output feature channel, and each input color channel. This adaptive weight is calculated based on the absolute deviation of the current pixel value from the global mean of its respective color channel, with channels having larger deviations receiving higher weights.

[0095] (3) Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region;

[0096] The computation space is adapted to a local enhancement factor that dynamically adjusts the gradient enhancement intensity based on the texture richness of the local region: applying a stronger enhancement effect to regions with complex textures and a weaker enhancement effect to regions with flat textures, thereby avoiding the amplification of noise.

[0097] (4) Combine the target multi-directional gradient magnitude, the target channel fusion weight and the target local enhancement factor, and generate the target detail enhancement feature map by non-linear activation operation and adding a bias term.

[0098] In one embodiment, see Figure 2 The present invention discloses a flowchart of a training method for an ancient calligraphy and painting image reconstruction model, which includes the following steps:

[0099] Step S201: Construct a training dataset for the reconstruction of ancient calligraphy and painting images.

[0100] Each sample in the training dataset is an ancient calligraphy and painting image pair, which includes: an original ancient calligraphy and painting image with varying degrees of degradation, and a high-quality, clear reference image corresponding to the original ancient calligraphy and painting image.

[0101] In each sample collection process, the selected ancient paintings and calligraphy artifacts are first digitally acquired using a high-resolution professional scanner or a professional digital camera equipped with a lens for capturing specific cultural assets, under strictly controlled lighting conditions, to obtain initial high-definition digital images as high-quality, clear reference images.

[0102] To simulate the complex degradation of calligraphy and paintings due to age in the real world and to construct ancient calligraphy and painting image pairs as samples, this application artificially synthesizes degradation processing on high-quality, clear reference images. The degradation processing strictly simulates multi-scale and non-uniform characteristics, including but not limited to simulating ink bleeding, blurring caused by paper fiber aging, uneven color fading in different areas, and dotted or patchy stains and mold spots, thereby obtaining original ancient calligraphy and painting images with degradation corresponding to the high-quality, clear reference images.

[0103] The original ancient paintings and calligraphy images and the high-quality, clear reference images are all in RGB (Red-Green-Blue) format.

[0104] Step S202: Perform multi-scale degradation simulation on the original ancient calligraphy and painting images in each sample of the training dataset, and use the degradation simulation results to perform adaptive degradation correction on the original ancient calligraphy and painting images to obtain preprocessed images.

[0105] This invention adaptively estimates the local degradation kernels of original ancient paintings and calligraphy images at different spatial locations and scales by performing multi-scale degradation simulation on the original images. Then, the estimated local degradation kernels are used to perform weighted fusion preprocessing on the original images to obtain preprocessed images. This process can effectively simulate and initially correct non-uniform degradation phenomena, enhance image detail and color information, and provide strong support for subsequent feature extraction.

[0106] Step S203: Perform blur detail feature enhancement processing on the preprocessed image, and extract the detail enhancement feature map from the enhanced preprocessed image.

[0107] The blurred details in ancient calligraphy and painting images are often hidden in the low-frequency components or textures of the image. The edge and texture signals are relatively weak. When processing such images, conventional edge detection operators or high-frequency filters not only have difficulty in effectively distinguishing between real details and noise, but also tend to amplify the noise introduced during the image degradation process, causing the detailed features to be submerged and unable to provide effective feature representation for subsequent reconstruction.

[0108] Based on this, the present invention performs blurred detail feature enhancement processing on the preprocessed image through an adaptive feature enhancement function. By combining local contrast adjustment and multi-directional gradient information fusion, blurred detail features are extracted and enhanced from the preprocessed image. This function can dynamically adjust the enhancement intensity according to the texture complexity of the local area and adaptively fuse gradient information from different color channels to generate a robust and detail-rich enhanced feature map.

[0109] Step S204: Based on the detailed enhancement feature map, the training dataset is trained using a supervised learning method to obtain the ancient calligraphy and painting image reconstruction model.

[0110] In one embodiment, step S202 may specifically include:

[0111] 1) For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, estimate the convolution kernel that represents the local blurring or degradation effect of the spatial location, and use it as the local degradation kernel.

[0112] The estimation of the local degradation kernel is achieved by minimizing the difference between the local image patch centered at the current location and corresponding scale, and the ideal sharp template. This ideal sharp template can be obtained from high-quality contemporaneous calligraphy and painting data, or it can be generated through prior knowledge of degradation. The expression for the local degradation kernel is shown in formula (1):

[0113] (1);

[0114] In the formula, Indicates spatial location The local degradation kernel estimated at the s-th scale is used to characterize the blurring or degradation effect of the corresponding local region at a specific scale;

[0115] x represents the row index in the image, used to locate the vertical position of the pixel;

[0116] y represents the column index in the image, used to locate the horizontal position of the pixel;

[0117] 's' represents the scale index, used to specify the observation scale for degradation modeling, and its value range is... ;

[0118] S represents the preset total number of scales, which is used to control the granularity of multiscale analysis;

[0119] Represents original ancient calligraphy and painting images in RGB format;

[0120] This represents the convolution kernel K that minimizes the subsequent loss function through optimization.

[0121] This indicates extracting location information from an image. Centered on, the Operations on local image patches defined by each scale;

[0122] Represents the two-dimensional convolution operator;

[0123] Indicates the location and the Ideal sharp image patch template at a given scale;

[0124] The L2 norm operator for vectors.

[0125] It should be noted that the value of each pixel in the original ancient calligraphy and painting images represents the light intensity or reflectance at that location in the red, green, and blue color channels. It is the input data for the preprocessing process.

[0126] It should be noted that the total scale number S is set according to the size range of typical degradation features in the painting and calligraphy image. For example, it can be set to 3 to model the degradation effects at different scales, such as fine brushstrokes, medium textures, and large faded areas.

[0127] 2) The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction to obtain a preprocessed image.

[0128] The calculation expression for the preprocessed image is shown in formula (2):

[0129] (2);

[0130] In the formula, Indicates preprocessed image In position The pixel value on the c-th color channel;

[0131] c represents the color channel index, and its value range is... These correspond to the red, green, and blue channels, respectively.

[0132] This indicates summation over all preset scales s;

[0133] Indicates the location At point s, the adaptive fusion weight corresponding to the s-th scale, i.e., the scale weight;

[0134] Indicates the use of degenerate kernel Local convolution is performed on the c-th color channel of the original ancient calligraphy and painting image to simulate the degradation effect at this scale;

[0135] Indicates the location The learnable bias term on the c-th color channel is used to correct color shifts or brightness deviations that may occur after image preprocessing.

[0136] Among them, scale weight Based on the variance calculation of the corresponding local image patch, regions with larger variances are usually assigned higher weights to retain more details, and the calculation method is shown in formula (3):

[0137] (3);

[0138] In the formula, Represents the natural exponential function;

[0139] This represents the scale-sensitive parameter corresponding to the s-th scale. It is an adjustable hyperparameter used to control the effect of local variance on the scale weights. The intensity of the impact;

[0140] This represents the variance operator, used to calculate the variance of pixel values ​​in a local image patch;

[0141] Indicated by position A local image patch defined at the s-th scale, centered on ;

[0142] The dummy index in the summation formula represents the scale and is used to calculate the normalized denominator for all weights.

[0143] Indicates the corresponding to the first The scale-sensitive parameters at each scale are adjustable hyperparameters used to control the effect of local variance on scale weights. The intensity of the impact;

[0144] Indicated by position Centered on, the A local image patch defined by a scale.

[0145] In one embodiment, step S203 may specifically include:

[0146] (1) For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude.

[0147] In practical applications, for each color channel of the preprocessed image, its gradient magnitude is calculated along four preset directions. This gradient magnitude represents the edge and texture variation information of the preprocessed image in different directions, specifically expressed as follows:

[0148] (4);

[0149] In the formula, Indicates the location of the preprocessed image. The c-th color channel, direction The gradient magnitude is used to characterize the rate of intensity change of the preprocessed image at that location along a specific direction;

[0150] Represents the gradient direction, with values ​​from a set. These represent horizontal, 45-degree diagonal, vertical, and 135-degree diagonal directions, respectively.

[0151] Indicates the direction Gradient operators, such as the Sobel operator.

[0152] In practice, the gradient magnitude calculation in the four directions is completed through convolution operations, and a specific gradient kernel is used to filter the image in each direction.

[0153] (2) Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel.

[0154] To adaptively fuse the gradient information of the three color channels, adaptive weights are calculated for each spatial location, each output feature channel, and each input color channel. These adaptive weights are calculated based on the absolute deviation of the current pixel value from the global mean of its corresponding color channel. Channels with larger deviations receive higher weights. The calculation method is shown in formula (5).

[0155] (5);

[0156] In the formula, Indicates the location At this point, an adaptive weight is used to fuse the gradient information of the input color channel c into the output feature channel d. The weight value is between 0 and 1.

[0157] d represents the channel index of the output detail feature map, and its value ranges from 0 to 1. ;

[0158] Indicates the total number of feature channels;

[0159] This represents a learnable weight adjustment parameter used to control the strength of the influence of pixel value differences in the input color channel c on the weights fused into the feature channel d.

[0160] This represents the absolute value operator;

[0161] This represents the global pixel mean of the c-th color channel of the preprocessed image across the entire dataset, used for centering.

[0162] Indicates the location of the preprocessed image. The pixel value on the c-th color channel;

[0163] This represents the dummy index in the summation formula, which represents the color channel and is used to calculate the normalized denominator for the weights.

[0164] Indicates the location of the preprocessed image. , No. Pixel values ​​on each color channel;

[0165] This indicates that the preprocessed image is on the entire dataset, the first... The global pixel mean of each color channel is used for centering.

[0166] (3) Calculate the local enhancement factor based on the texture complexity and detail richness of the local region.

[0167] The computation space is adapted to a local enhancement factor that dynamically adjusts the gradient enhancement intensity based on the texture richness of the local region: applying a stronger enhancement effect to regions with complex textures and a weaker enhancement effect to regions with flat textures, thereby avoiding the amplification of noise.

[0168] The expression for the local enhancement factor is as follows:

[0169] (6);

[0170] In the formula, Indicates the location The local enhancement factor at that location is greater than or equal to scalar;

[0171] This represents the global enhancement intensity parameter, which is an adjustable hyperparameter used to control the overall enhancement magnitude;

[0172] Represents the natural logarithm function;

[0173] Indicated by position The standard deviation of pixel values ​​within a local neighborhood centered on the pixel is used to measure the texture complexity and detail richness of that local region.

[0174] (4) Combine the multi-directional gradient magnitude, the channel fusion weight and the local enhancement factor, and generate a detail enhancement feature map by non-linear activation operation and adding a bias term.

[0175] The expression for the detail-enhancing feature map is as follows:

[0176] (7);

[0177] In the formula, This indicates the location of the output detail-enhanced feature map. The value on the feature channel d;

[0178] This represents the modified linear unit activation function, used to introduce nonlinearity and filter out possible negative responses;

[0179] This represents the set of four preset gradient directions. Perform summation;

[0180] This represents the learnable spatial bias term corresponding to feature channel d, used to adjust the baseline of the feature map.

[0181] In one embodiment, step S204 may specifically include:

[0182] (1) The detail enhancement feature map is processed using the detail enhancement convolution module to obtain the detail enhancement value.

[0183] Conventional convolutional neural networks, due to their fixed receptive field and single convolution mode, may not be able to effectively capture multi-scale details hidden in images when processing blurry details in ancient paintings and calligraphy. This results in some scale details being ignored or smoothed out during propagation.

[0184] Based on this, the present invention constructs a detail enhancement convolution module, which captures multi-scale contextual information of detail enhancement feature maps through parallel dilated convolution with multiple dilation rates, and introduces a spatially adaptive attention mechanism to dynamically fuse feature responses at different scales, thereby enhancing the network's ability to perceive blurred and scale-variable details in calligraphy and painting.

[0185] The process of obtaining detail enhancement values ​​using the detail enhancement convolution module specifically includes:

[0186] 1) Calculate a set of weights for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates.

[0187] For each spatial location and each output channel, corresponding attention weights are calculated. These weights measure the importance of different dilation rates of the dilated convolutional branches for the specific features of that spatial location and output channel. Specifically, the attention score for each dilated convolutional branch is first calculated, and then these scores are normalized using the Softmax function, as follows:

[0188] (8);

[0189] In the formula, This represents the attention weights of the dilation-rate dilated convolution branch, i.e., at position. , corresponding to the normalized attention weights of the r-th expansion rate and the o-th output channel;

[0190] r represents the expansion rate index, and its value range is... ;

[0191] R represents the total preset expansion rate;

[0192] o represents the module output channel index, with a value range of 1. ;

[0193] Indicates the total number of output channels of the module;

[0194] Indicates the location The attention score corresponds to the r-th expansion rate and the o-th output channel. The higher the score, the greater the contribution of the scale branch at that position to the formation of the o-th output channel feature.

[0195] Let represent a set of learnable parameters used to generate attention scores, which establishes the spatial correlation between the input feature channel d and the output channel o at the r-th expansion rate;

[0196] This represents the dummy index in the summation formula, which represents the expansion rate and is used to calculate the normalized denominator of the weights.

[0197] Let represent a set of learnable parameters used to generate attention scores, which establishes the relationship between the input feature channel d and the output channel o at the th... Spatial correlation under an expansion rate.

[0198] Among them, attention score The calculation formula is as follows:

[0199] (9);

[0200] In the formula, This represents summing over all output channels of the input detail-enhancing feature map;

[0201] Let represent a set of learnable parameters used to generate attention scores, which establishes the spatial correlation between the input feature channel d and the output channel o at the r-th expansion rate;

[0202] This indicates the location of the output detail-enhanced feature map. The value on the feature channel d.

[0203] 2) Perform parallel convolution operations on the detail enhancement feature map using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates.

[0204] Multiple dilated convolutional kernels with different dilation rates are used to perform convolution operations in parallel on the input detail enhancement feature map. These kernels share the same spatial size but have different dilation rates, thus enabling the acquisition of receptive fields of different scales without increasing the number of parameters. This operation outputs a three-dimensional tensor, specifically represented as:

[0205] (10);

[0206] In the formula, The dilated convolution branch with the r-th dilation rate is located at position r. The convolution result on the o-th output channel;

[0207] This represents a dilated convolution operation with an expansion rate of r, which expands the receptive field by inserting zero values ​​between the weights of a standard convolution kernel.

[0208] The weight parameter corresponding to the o-th output channel in the dilated convolution kernel with the r-th dilation rate;

[0209] This indicates the location of the input detail-enhanced feature map. All channel values ​​at that location.

[0210] 3) Multiply the convolution results of the dilated convolution branches with the corresponding attention weights respectively, sum them, and add the summation result to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0211] (11);

[0212] In the formula, This indicates that the detail-enhancing convolutional module is located at... The detail enhancement value of the o-th output channel;

[0213] This represents summing the results of all R dilated convolution branches with different dilation rates;

[0214] This represents the learnable bias parameter corresponding to the o-th output channel, used to adjust the baseline of the output characteristics.

[0215] (2) Based on the detail enhancement value output by the detail enhancement convolution module, a reconstructed clear calligraphy and painting image is generated.

[0216] Conventional image reconstruction networks such as U-Net and its variants typically use mean squared error or L1 loss function for supervised training. These loss functions focus on pixel-level average accuracy, but they can easily lead to overly smooth reconstruction results, losing the sharpness of brushstrokes and the layering of ink tones unique to calligraphy and painting images, making it difficult to meet the strict requirements for restoring the characteristics of calligraphy and painting art.

[0217] This invention constructs a multi-stage cyclic refinement reconstruction network and introduces a quaternary loss function. This multi-stage cyclic refinement reconstruction network first deeply fuses the detail enhancement values ​​output by the detail enhancement convolution module with the preprocessed image to generate an initial reconstructed image. Then, iterative detail refinement is performed on the initial result through a cyclic feedback structure. The introduced quaternary loss function not only considers pixel-level accuracy but also innovatively introduces a consistency loss based on multi-scale statistical features of the original preprocessed image, forcing the reconstructed result to maintain consistency with the original calligraphy and painting in terms of texture style and overall visual perception. The specific steps are as follows:

[0218] 1) The detail enhancement map, which is composed of the detail enhancement values ​​output by the detail enhancement convolution module, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstruction image.

[0219] The initial reconstructed image, which integrates low-level color information and high-level multi-scale detail features, is represented as follows:

[0220] (12);

[0221] In the formula, This represents the RGB pixel vector at row u and column v of the final reconstructed clear image of the calligraphy and painting.

[0222] u represents the row index in the initial reconstructed image space, used to locate the vertical position of the pixel;

[0223] v represents the column index in the initial reconstructed image space, used to locate the horizontal position of the pixel;

[0224] The parameter is The initial reconstruction subnetwork mapping function consists of convolutional layers, nonlinear activation functions, and upsampling operations.

[0225] This indicates a splicing operation along the channel dimension;

[0226] This represents the vector consisting of all output feature channel values ​​at row u and column v of the detail enhancement convolution module;

[0227] Indicates preprocessed image A vector consisting of the RGB three-channel values ​​at the u-th row and v-th column;

[0228] This represents the set of all trainable parameters, such as weights and biases, in the initial reconstructed subnetwork.

[0229] In the specific implementation, the range of values ​​for the row index u and column index v is consistent with the size of the clearly reconstructed ancient calligraphy and painting image. If the clearly reconstructed ancient calligraphy and painting image is H×W pixels, then... , .

[0230] 2) Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and a fully connected network is combined to generate a reconstructed clear calligraphy and painting image.

[0231] The entire process is represented as follows:

[0232] (13);

[0233] In the formula, Reconstructing clear images of calligraphy and paintings The RGB pixel vector at row u and column v;

[0234] The parameter is The fully connected network is specifically implemented using two fully connected layers;

[0235] This represents the encoding model of an autoencoder;

[0236] This represents the decoding model of an autoencoder;

[0237] This represents the set of trainable parameters for a fully connected network.

[0238] (3) Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed.

[0239] The expression for the quaternary loss function is as follows:

[0240] (14);

[0241] In the formula, The quaternion loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local stroke coherence loss, and rhythm consistency loss.

[0242] This represents the balancing weight of the pixel loss term, with an example value of 1.0.

[0243] This represents the balancing weight of the perceptual loss term, with an example value of 0.1.

[0244] This represents the balancing weight of the stroke loss term, with an example value of 0.5.

[0245] This represents the balancing weight of the loss of spirit and charm, with an example value of 0.2.

[0246] This represents the pixel fidelity loss, specifically employing a smoothed L1 loss to enhance robustness against outliers. Its calculation method is as follows:

[0247] (15);

[0248] In the formula, N represents the total number of pixels in a single image;

[0249] c represents the color channel index, with a value range of {1, 2, 3}, corresponding to the red, green, and blue channels respectively;

[0250] Representing authentic and clear images of ancient calligraphy and paintings. In the line, number Column, No. The pixel values ​​of each channel;

[0251] This represents the smoothed L1 loss function.

[0252] The multi-scale perceptual loss is represented by comparing the feature differences between the reconstructed image and the real image at multiple layers of the pre-trained VGG network. Its calculation method is expressed as follows:

[0253] (16);

[0254] In the formula, I represents the selected feature layer index;

[0255] L represents the set of selected feature layers, for example {'relu1_2', 'relu2_2', 'relu3_3', 'relu4_3'};

[0256] This indicates the pre-trained VGG network's... Feature extraction function of the layer;

[0257] This indicates the pre-trained VGG network's... Layer-by-layer reconstruction of clear calligraphy and painting images Feature extraction function;

[0258] This indicates the pre-trained VGG network's... Layers of realistic and clear images of ancient calligraphy and paintings Feature extraction function;

[0259] Indicates the first The height of the layer feature map;

[0260] Indicates the first Width of the layer feature map;

[0261] Indicates the first The number of channels in the layer feature map.

[0262] This represents the loss of local stroke coherence, emphasizing the consistency between the reconstructed image and the real image in the distribution of local gradient magnitudes, in order to maintain the sharpness of stroke edges. Its calculation method is expressed as follows:

[0263] (17);

[0264] In the formula, Reconstructing clear images of calligraphy and paintings In the line, number Gradient magnitude at column;

[0265] Representing authentic and clear images of ancient calligraphy and paintings. In the line, number The gradient magnitude at the column.

[0266] To represent the loss of stylistic consistency, and to ensure that the reconstructed image remains faithful to the original in terms of overall texture style statistics, the Gram matrix difference between the reconstructed image and the upsampled original preprocessed image on multi-scale features is calculated. The calculation method is as follows:

[0267] (18);

[0268] In the formula, This represents the set of feature layers used for style comparison, for example, {'relu2_2', 'relu3_3', 'relu4_3'};

[0269] This indicates a bicubic interpolation upsampling operation, which scales the image to match the reconstructed clear image of the calligraphy or painting. Same size;

[0270] This indicates that for dimension 1 The Gram matrix calculated from the feature map F1 of the l-th layer;

[0271] p represents the row index of the Gram matrix, corresponding to the feature channel, and its value ranges from 1 to 2. ;

[0272] This represents the number of channels in the feature map of the l-th layer;

[0273] This indicates the pre-trained VGG network's... Layer to the first Layer-by-layer reconstruction of clear calligraphy and painting images Gram matrix calculated from features extracted from authentic and clear ancient calligraphy and painting images;

[0274] This indicates the pre-trained VGG network's... Layer-by-layer preprocessed image Gram matrix calculated from features extracted by bicubic interpolation upsampling;

[0275] This represents the Frobenius norm of the matrix.

[0276] In one embodiment, the feature quality of different methods at different feature extraction stages is compared. The experiment compares the image reconstruction method proposed in this invention with four conventional techniques: global normalization, histogram equalization, a super-resolution reconstruction method based on convolutional neural networks, and the U-Net deep learning reconstruction network. The experiment scores the image reconstruction of ancient calligraphy and painting images from four feature dimensions crucial to their reconstruction: edge preservation (assessing the sharpness of brushstroke contours), texture sharpness (assessing the clarity of paper fibers and ink texture), color fidelity (assessing the accuracy of color reproduction), and detail richness (assessing the degree of preservation of subtle brushstrokes and faded area details). The experiment uses the same test dataset, and professional researchers blindly score the reconstruction results to ensure objectivity. Figure 3 The bar chart showing the quality comparison of different methods at each feature extraction stage reveals that the image reconstruction method disclosed in this invention achieved the highest scores across all four feature dimensions, with particularly significant advantages in detail richness and color fidelity. The two traditional methods (global normalization and histogram equalization) scored lower across all features because their globally uniform processing cannot adapt to the non-uniform degradation characteristics of ancient calligraphy and paintings. The two deep learning methods performed reasonably well on some features, but were significantly lacking in detail richness. Experimental results demonstrate that the image reconstruction method disclosed in this invention, through its adaptive detail enhancement function and multi-scale attention mechanism, can more comprehensively and accurately restore and enhance various artistic features of ancient calligraphy and paintings.

[0277] In one embodiment, the performance of the image reconstruction method disclosed in this invention is analyzed at different scale numbers, investigating the impact of the key hyperparameter—scale number—of the multi-scale degradation modeling module on the overall performance. In the experimental setup, all other parameters were fixed, and only the scale number was systematically changed. Four key metrics were evaluated: peak signal-to-noise ratio (PSNR) (an objective indicator of reconstruction quality), structural similarity (a perceptual quality indicator), edge preservation (a brushstroke detail indicator), and processing time (an efficiency indicator). The units for the first three metrics are decibels, dimensionless, and dimensionless, respectively; higher values ​​indicate better performance. The unit for processing time is seconds.

[0278] Figure 4 The graph shows the peak signal-to-noise ratio as a function of scale number. Peak signal-to-noise ratio (PSNR) is an objective indicator of reconstruction quality. Figure 5 The graph shows the variation of structural similarity with scale number; Figure 6 The graph shows the edge preservation degree as a function of scale number; Figure 7 The graph shows the processing time as a function of the number of scales.

[0279] from Figures 4-7 As can be seen, with the increase of the number of scales, the peak signal-to-noise ratio, structural similarity, and edge preservation all rise rapidly at first, approaching their peak at a scale of 4 or 5, and then slowly decline or tend to plateau. Processing time, however, increases approximately linearly with the number of scales. Experimental results show that a larger scale is not always better: too few scales prevent the model from fully capturing multi-scale degradation features, leading to poor performance; too many scales, while potentially capturing finer-grained features, significantly increase computational complexity and may introduce redundancy or noise, resulting in limited or even decreased performance improvement. Therefore, the existence of an optimal scale range provides a clear experimental basis for the selection of the scale for multi-scale modeling in this invention, ensuring the best balance between effectiveness and efficiency.

[0280] (4) For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is adopted for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

[0281] In this study, supervised learning was used to update all training parameters in the ancient calligraphy and painting image reconstruction model.

[0282] Specifically, in each iteration, original ancient calligraphy and painting images with varying degrees of degradation from the training dataset are input into the ancient calligraphy and painting image reconstruction model to obtain the reconstructed clear calligraphy and painting images output by the model. A quaternary loss function is calculated based on the reconstructed clear calligraphy and painting images and the corresponding high-quality clear reference images from the training dataset. Based on the calculation results of the quaternary loss function, the gradient of each learnable parameter in the ancient calligraphy and painting image reconstruction model is adjusted using the backpropagation algorithm. This process is iterated continuously until the quaternary loss function is minimized.

[0283] After each training cycle, the quaternary loss function value is calculated on the validation set using the current model parameters. The stopping condition is set based on the validation set performance, specifically employing an "early stopping" strategy. During this strategy, the quaternary loss function result calculated on the validation set is continuously monitored. If the quaternary loss function value does not decrease within five consecutive training cycles, the model performance is considered to have reached saturation or overfitting has begun. At this point, the stopping condition is triggered, terminating the training cycle. Finally, the ancient calligraphy and painting image reconstruction model from the last iteration is selected as the completed ancient calligraphy and painting image reconstruction model.

[0284] Corresponding to the above method embodiments, the present invention discloses an image reconstruction apparatus.

[0285] See Figure 8 The present invention discloses a schematic diagram of an image reconstruction apparatus, which may include:

[0286] Image acquisition unit 301 is used to acquire images of ancient paintings and calligraphy to be reconstructed.

[0287] Among them, the ancient calligraphy and painting images to be reconstructed refer to new, unprocessed ancient calligraphy and painting images.

[0288] The preprocessing unit 302 is used to perform multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results, so as to obtain the preprocessed image to be reconstructed.

[0289] Ancient paintings and calligraphy images often exhibit degradation phenomena such as blurring, loss of detail, and color fading due to their age. This type of degradation is multi-scale and spatially non-uniform, meaning that the degree and pattern of degradation vary in different regions at different scales. Conventional techniques such as global normalization or histogram equalization, which assume that the degradation process is uniform across the entire image, are difficult to effectively handle this complex non-uniform degradation. This often leads to over-processing of detailed areas or under-processing of smooth areas, failing to provide high-quality preprocessing data for subsequent reconstruction stages.

[0290] This invention performs multi-scale degradation simulation on original ancient calligraphy and painting images, and uses the degradation simulation results to perform adaptive degradation correction on the ancient calligraphy and painting images to be reconstructed. This can effectively handle the non-uniform degradation phenomenon of images in space and scale, enhance the detail and color information of the images, and provide strong support for subsequent feature extraction work.

[0291] The feature enhancement unit 303 is used to perform blurred detail feature enhancement processing on the preprocessed image to be reconstructed, and extract the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed.

[0292] The blurred details in ancient calligraphy and painting images are often hidden in the low-frequency components or textures of the image. The edge and texture signals are relatively weak. When processing such images, conventional edge detection operators or high-frequency filters not only have difficulty in effectively distinguishing between real details and noise, but also tend to amplify the noise introduced during the image degradation process, causing the detailed features to be submerged and unable to provide effective feature representation for subsequent reconstruction.

[0293] Based on this, the present invention enhances the blurred details of the preprocessed image to be reconstructed by performing blurred detail feature enhancement processing. This can effectively suppress noise interference while enhancing blurred details in a targeted manner. While preserving the overall structural information of the image, it improves the recognizability of weak details, thus laying a solid foundation for the accurate reconstruction of ancient calligraphy and painting images. This replaces the edge detection or high-frequency filtering operations of conventional methods, thereby effectively solving the problem of destroying the original artistic brushstrokes in conventional methods.

[0294] Image reconstruction unit 304 is used to input the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, and obtain a reconstructed clear ancient calligraphy and painting image.

[0295] The ancient calligraphy and painting image reconstruction model is as follows: the training dataset for ancient calligraphy and painting image reconstruction is trained using a supervised learning method, with the minimization of the quaternary loss function as the training objective. The model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

[0296] It should be noted that the quaternary loss function in this invention is a loss function that combines pixel fidelity loss, multi-scale perception loss, local stroke coherence loss, and rhythm consistency loss.

[0297] Among them, pixel fidelity loss can ensure that the reconstructed ancient calligraphy and painting images are as close as possible to the original images at the pixel level.

[0298] Multi-scale perceptual loss evaluates and optimizes images at different scales. At the macro scale, it ensures that the overall composition and layout of the reconstructed image are consistent with the original painting or calligraphy; at the micro scale, it focuses on the presentation of details, such as the thickness of brushstrokes and the variation in ink density. This multi-scale consideration ensures that the reconstructed image is highly similar to the original work at all levels, thus improving the overall image quality.

[0299] The brushstrokes in ancient Chinese calligraphy and painting are a crucial manifestation of their artistic style. Each stroke, from its beginning to its execution, embodies the artist's emotions and skill. The loss of continuity in local brushstrokes ensures the smoothness and consistency of the strokes during reconstruction, avoiding issues such as broken or disjointed strokes. For example, in reconstructing calligraphy, this ensures natural transitions and even pressure in the strokes, allowing the reconstructed work to retain its rhythm and flow.

[0300] The spirit and rhythm of calligraphy and painting are the soul of ancient Chinese art, embodying the overall artistic conception and appeal of the work. The quaternary loss function incorporates the loss of spirit and rhythm consistency, analyzing the spirit and rhythm characteristics of the original work, such as the fluidity of lines, the harmony of colors, and the appropriate density of composition, to preserve and inherit these spirit and rhythm as much as possible during the reconstruction process. This ensures that the reconstructed ancient calligraphy and painting not only resembles the original in appearance but also inherits its artistic essence, allowing viewers to experience the unique charm inherent in ancient calligraphy and painting.

[0301] Therefore, the quaternary loss function provides a comprehensive evaluation metric for training ancient calligraphy and painting reconstruction models. It comprehensively considers multiple aspects of the image, enabling the model to optimize from different angles during training and avoiding the local optima problem that may occur with a single loss function. By minimizing the quaternary loss function, the model can learn more comprehensive and accurate image features, thereby improving the accuracy and quality of reconstruction. It effectively overcomes the limitations of traditional U-Net and other networks with their single network structure and loss functions that only focus on pixel precision, achieving the collaborative reconstruction of artistic details and stylistic characteristics of calligraphy and painting images.

[0302] In summary, this invention discloses an image reconstruction device that acquires an ancient painting or calligraphy image to be reconstructed, performs multi-scale degradation simulation on the image, uses the degradation simulation results to perform adaptive degradation correction on the image to be reconstructed to obtain a preprocessed image to be reconstructed, performs blurry detail feature enhancement processing on the preprocessed image to be reconstructed, extracts a target detail enhancement feature map from the enhanced preprocessed image to be reconstructed, and inputs the target detail enhancement feature map into a pre-trained ancient painting or calligraphy image reconstruction model for painting or calligraphy image reconstruction processing to obtain a reconstructed clear ancient painting or calligraphy image. This invention simulates the degradation of original ancient calligraphy and painting images at multiple scales and uses the simulation results to adaptively correct the degradation of the images to be reconstructed. This effectively addresses the non-uniform degradation phenomena in space and scale, enhancing the detail and color information of the images. By enhancing the blurred detail features of the preprocessed images to be reconstructed, it effectively suppresses noise interference while selectively enhancing blurred details, enhancing subtle artistic details while preserving the overall structural information of the image. Furthermore, by introducing a quaternary loss function that integrates pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss during the training of the ancient calligraphy and painting image reconstruction model, it achieves the goal of improving image clarity while maintaining the original brushwork spirit and aesthetic characteristics.

[0303] In one embodiment, the preprocessing unit 302 can specifically be used for:

[0304] For the ancient calligraphy and painting image to be reconstructed, at each spatial location and preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel;

[0305] The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales;

[0306] The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image to be reconstructed.

[0307] In one embodiment, the feature enhancement unit 303 can specifically be used for:

[0308] For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude;

[0309] Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated.

[0310] Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region;

[0311] The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map.

[0312] In one embodiment, the image reconstruction apparatus may further include:

[0313] The training set construction unit is used to construct a training dataset for the reconstruction of ancient calligraphy and painting images. Each sample in the training dataset is an ancient calligraphy and painting image pair, which includes: an original ancient calligraphy and painting image with varying degrees of degradation, and a high-quality, clear reference image corresponding to the original ancient calligraphy and painting image.

[0314] The degradation correction unit is used to perform multi-scale degradation simulation on the original ancient calligraphy and painting images in each sample of the training dataset, and to perform adaptive degradation correction on the original ancient calligraphy and painting images using the degradation simulation results to obtain a preprocessed image.

[0315] An enhancement processing unit is used to perform blurred detail feature enhancement processing on the preprocessed image and extract the detail enhancement feature map from the enhanced preprocessed image;

[0316] The model training unit is used to train the training dataset using a supervised learning method based on the detailed enhancement feature map to obtain the ancient calligraphy and painting image reconstruction model.

[0317] In one embodiment, the degradation correction unit can be specifically used for:

[0318] For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel;

[0319] The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image.

[0320] In one embodiment, the enhancement processing unit may specifically be used for:

[0321] For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude;

[0322] Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel;

[0323] Calculate the local enhancement factor based on the texture complexity and detail richness of the local region;

[0324] The multi-directional gradient magnitude, the channel fusion weight, and the local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate a detail-enhanced feature map.

[0325] In one embodiment, the model training unit can be specifically used for:

[0326] The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values;

[0327] A reconstructed, clear image of calligraphy and painting is generated based on the aforementioned detail enhancement values;

[0328] Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed.

[0329] For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is used for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

[0330] In one embodiment, the model training unit can also be used for:

[0331] A set of weights is calculated for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates.

[0332] The detail enhancement feature map is subjected to parallel convolution operations using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates;

[0333] The convolution results of the dilated convolution branches with different dilation rates are multiplied by their corresponding attention weights and summed. The summation result is then added to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0334] In one embodiment, the model training unit can also be used for:

[0335] The detail enhancement map, composed of the aforementioned detail enhancement values, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstructed image.

[0336] Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and the reconstructed clear calligraphy and painting image is generated by combining a fully connected network.

[0337] It should be noted that for the specific working principles of each component in the device embodiment, please refer to the corresponding section of the method embodiment, which will not be repeated here.

[0338] Corresponding to the above embodiments, the present invention also discloses a computer storage medium that stores at least one instruction, which, when executed by a processor, implements the steps shown in the embodiments of the image reconstruction method.

[0339] Corresponding to the above embodiments, such as Figure 9 As shown, the present invention also provides a schematic diagram of the structure of an electronic device, which may include: a processor 1 and a memory 2;

[0340] The processor 1 and memory 2 communicate with each other via communication bus 3.

[0341] Processor 1, for executing at least one instruction;

[0342] Memory 2 is used to store at least one instruction;

[0343] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0344] Memory 2 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0345] The processor executes at least one instruction to implement the steps shown in the embodiment of the image reconstruction method.

[0346] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0347] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0348] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of image reconstruction, characterized by, include: Obtain images of ancient paintings and calligraphy to be reconstructed; For the ancient calligraphy and painting image to be reconstructed, at each spatial location and preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales; The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in a preprocessed image to be reconstructed. For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated. Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region; The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map. The target detail enhancement feature map is input into a pre-trained ancient calligraphy and painting image reconstruction model for calligraphy and painting image reconstruction processing to obtain a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met, with the minimization of the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

2. The image reconstruction method of claim 1, wherein, The training process of the ancient calligraphy and painting image reconstruction model includes: A training dataset for reconstructing ancient calligraphy and painting images is constructed, wherein each sample in the training dataset is an ancient calligraphy and painting image pair, and the ancient calligraphy and painting image pair includes: an original ancient calligraphy and painting image with different degrees of degradation, and a high-quality clear reference image corresponding to the original ancient calligraphy and painting image; Multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and the degradation simulation results are used to adaptively correct the degradation of the original ancient calligraphy and painting images to obtain preprocessed images; The preprocessed image is subjected to blurry detail feature enhancement processing, and the detail enhancement feature map is extracted from the enhanced preprocessed image; Based on the enhanced detail feature maps, the training dataset is trained using a supervised learning method to obtain the ancient calligraphy and painting image reconstruction model.

3. The image reconstruction method of claim 2, wherein, Multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and adaptive degradation correction is performed on the original ancient calligraphy and painting images using the degradation simulation results to obtain preprocessed images, including: For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image.

4. The image reconstruction method of claim 2 or 3, characterized in that, The preprocessed image is subjected to blurry detail feature enhancement processing, and the detail enhancement feature map is extracted from the enhanced preprocessed image, including: For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel; Calculate the local enhancement factor based on the texture complexity and detail richness of the local region; The multi-directional gradient magnitude, the channel fusion weight, and the local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate a detail-enhanced feature map.

5. The image reconstruction method of claim 2, wherein, Based on the aforementioned detail enhancement feature maps, a supervised learning method is used to train the training dataset to obtain the ancient calligraphy and painting image reconstruction model, including: The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values; A reconstructed, clear image of calligraphy and painting is generated based on the aforementioned detail enhancement values; Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed. For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is used for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

6. The image reconstruction method of claim 5, wherein, The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values, including: A set of weights is calculated for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates. The detail enhancement feature map is subjected to parallel convolution operations using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates; The convolution results of the dilated convolution branches with different dilation rates are multiplied by their corresponding attention weights and summed. The summation result is then added to the bias term of the corresponding output channel to obtain the detail enhancement value.

7. The image reconstruction method according to claim 5, characterized in that, Based on the aforementioned detail enhancement values, a reconstructed clear image of the calligraphy or painting is generated, including: The detail enhancement map, composed of the aforementioned detail enhancement values, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstructed image. Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and the reconstructed clear calligraphy and painting image is generated by combining a fully connected network.

8. An image reconstruction apparatus, characterized in that, include: The image acquisition unit is used to acquire images of ancient paintings and calligraphy to be reconstructed. The preprocessing unit is used to estimate, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location for the ancient calligraphy and painting image to be reconstructed, and to use it as the local degradation kernel. The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales; The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in a preprocessed image to be reconstructed. The feature enhancement unit is used to calculate the target multi-directional gradient magnitude for each color channel of the preprocessed image to be reconstructed; Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated; the target local enhancement factor is calculated based on the texture complexity and detail richness of the local region. The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map. The image reconstruction unit is used to input the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, thereby obtaining a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is trained by continuously optimizing the model parameters until the stopping iteration condition is met, with minimizing the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

9. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which, when executed by a processor, implements the image reconstruction method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes: a memory and a processor; The memory is used to store at least one instruction; The processor is used to execute the at least one instruction to implement the image reconstruction method as described in any one of claims 1 to 7.

Images