Image reconstruction method and related device

Abstract

The invention discloses an image reconstruction method and a related device, and relates to the field of image processing, and the method comprises the steps: obtaining a to-be-reconstructed ancient painting and calligraphy image, carrying out the multi-scale degradation simulation of the to-be-reconstructed ancient painting and calligraphy image, carrying out the self-adaptive degradation correction of the to-be-reconstructed ancient painting and calligraphy image through a degradation simulation result, and obtaining a preprocessed to-be-reconstructed image; and performing fuzzy detail feature enhancement processing on the preprocessed to-be-reconstructed image, extracting a target detail enhancement feature map from the preprocessed to-be-reconstructed image after enhancement processing, and inputting the target detail enhancement feature map into a pre-trained ancient painting and calligraphy image reconstruction model for painting and calligraphy image reconstruction processing to obtain a reconstructed clear ancient painting and calligraphy image. According to the method, self-adaptive degradation correction is carried out on the original ancient painting and calligraphy image, so that the non-uniform degradation phenomenon of the image is effectively processed; noise is suppressed and fuzzy details are enhanced through fuzzy detail feature enhancement processing; a quaternary loss function is introduced during ancient painting and calligraphy image reconstruction model, so that the original writing and ink romantic charm is kept while the image definition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and more particularly, to an image reconstruction method and related device. BACKGROUND

[0002] In the field of cultural heritage digitization, high-quality restoration of ancient painting and calligraphy images is facing severe challenges. After hundreds of years of natural aging, painting and calligraphy generally exhibit complex degradation phenomena such as ink spreading, fiber breakage, and pigment fading. These degradation phenomena are not only uneven in spatial distribution, but also exhibit multi-scale characteristics in their manifestations.

[0003] Currently, there are two main image reconstruction methods. One method is to use image enhancement techniques (such as histogram equalization) to reconstruct ancient painting and calligraphy images. However, image enhancement techniques adopt a global uniform processing approach, which is difficult to adapt to the non-uniform degradation characteristics of painting and calligraphy in space and scale, often leading to over-processing in the detail areas of painting and calligraphy images, while the smooth areas are not processed enough. The other method is a deep learning-based super-resolution method, which can improve the overall visual clarity of ancient painting and calligraphy images, but it generally ignores the unique brush strokes and ink color of painting and calligraphy, resulting in the problem of artistic distortion of the reconstruction results being clear but losing the spirit.

[0004] Therefore, how to provide an image reconstruction method that can effectively handle the non-uniform degradation of ancient painting and calligraphy images, while enhancing weak artistic details and maintaining the original artistic spirit, has become a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, how to provide an image reconstruction method that can effectively handle the non-uniform degradation of ancient painting and calligraphy images, while enhancing weak artistic details and maintaining the original artistic spirit, has become a technical problem that needs to be solved by those skilled in the art.

[0006] An image reconstruction method, comprising: obtaining an ancient painting and calligraphy image to be reconstructed; performing multi-scale degradation simulation on the ancient painting and calligraphy image to be reconstructed, and using the degradation simulation result to perform adaptive degradation correction on the ancient painting and calligraphy image to be reconstructed, to obtain a preprocessed image to be reconstructed; performing blur detail feature enhancement processing on the preprocessed image to be reconstructed, and extracting a target detail enhancement feature map from the preprocessed image to be reconstructed after enhancement processing; inputting the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, to obtain a reconstructed clear ancient calligraphy and painting image, wherein the ancient calligraphy and painting image reconstruction model is obtained by continuously optimizing model parameters to meet a stop iteration condition, with the minimum of a four-element loss function as a training target, and the four-element loss function is a loss function combining a pixel fidelity loss, a multi-scale perception loss, a local stroke coherence loss, and a qi and rhythm consistency loss.

[0007] Optionally, the ancient calligraphy and painting image to be reconstructed is subjected to multi-scale degradation simulation, and the degradation simulation result is used to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed, to obtain a preprocessed image to be reconstructed, including: For the ancient calligraphy and painting image to be reconstructed, a convolution kernel representing local blur or degradation effect of a spatial position is estimated at each spatial position and a preset scale, and is used as a local degradation kernel; The local degradation kernel is used to perform degradation simulation on the ancient calligraphy and painting image to be reconstructed at different scales; The simulation results at different scales are adaptively fused according to the detail richness of each local region, and a bias term is added to adjust the color to perform adaptive degradation correction, to obtain the preprocessed image to be reconstructed.

[0008] Optionally, the preprocessed image to be reconstructed is subjected to blur detail feature enhancement processing, and a target detail enhancement feature map is extracted from the preprocessed image to be reconstructed after enhancement processing, including: For each color channel of the preprocessed image to be reconstructed, a target multi-directional gradient amplitude is calculated; Based on the absolute deviation of a current pixel value of the preprocessed image to be reconstructed from a global average value of a color channel to which the current pixel value belongs, a target channel fusion weight corresponding to each color channel is calculated; A target local enhancement factor is calculated according to the texture complexity and the detail richness of a local region; The target multi-directional gradient amplitude, the target channel fusion weight, and the target local enhancement factor are combined, a nonlinear activation operation is performed, and a bias term is added, to generate the target detail enhancement feature map.

[0009] Optionally, the training process of the ancient calligraphy and painting image reconstruction model includes: A training data set for ancient calligraphy and painting image reconstruction is constructed, wherein each sample in the training data set is an ancient calligraphy and painting image pair, and the ancient calligraphy and painting image pair includes an original ancient calligraphy and painting image with different degrees of degradation and a high-quality clear reference image corresponding to the original ancient calligraphy and painting image; performing multi-scale degradation simulation on the original ancient calligraphy and painting image in each sample of the training data set, and performing adaptive degradation correction on the original ancient calligraphy and painting image by using the degradation simulation result to obtain a preprocessed image; performing blur detail feature enhancement processing on the preprocessed image, and extracting a detail enhancement feature map from the preprocessed image after enhancement processing; training the training data set in a supervised learning manner based on the detail enhancement feature map to obtain the ancient calligraphy and painting image reconstruction model.

[0010] Optionally, the multi-scale degradation simulation is performed on the original ancient calligraphy and painting image in each sample of the training data set, and the adaptive degradation correction is performed on the original ancient calligraphy and painting image by using the degradation simulation result to obtain a preprocessed image, including: For each of the original ancient calligraphy and painting images, a convolution kernel representing local blur or degradation effect at each spatial position and a predetermined scale is estimated and used as a local degradation kernel; The local degradation kernel is used to perform degradation simulation on the original ancient calligraphy and painting image at different scales, and the simulation results at different scales are adaptively fused according to the detail richness of each local region, and a bias term is added to adjust the color for adaptive degradation correction to obtain the preprocessed image.

[0011] Optionally, the blur detail feature enhancement processing is performed on the preprocessed image, and the detail enhancement feature map is extracted from the preprocessed image after enhancement processing, including: For each color channel of the preprocessed image, a multi-directional gradient amplitude is calculated; Based on the absolute deviation of the current pixel value of the preprocessed image and the global mean value of the color channel to which the current pixel value belongs, a channel fusion weight corresponding to each color channel is calculated; A local enhancement factor is calculated according to the texture complexity and detail richness of the local region; The multi-directional gradient amplitude, the channel fusion weight, and the local enhancement factor are combined, and a bias term is added through a nonlinear activation operation to generate a detail enhancement feature map.

[0012] Optionally, the training data set is trained in a supervised learning manner based on the detail enhancement feature map to obtain the ancient calligraphy and painting image reconstruction model, including: The detail enhancement feature map is processed by using a detail enhancement convolution module to obtain a detail enhancement value; A reconstructed clear calligraphy and painting image is generated based on the detail enhancement value; A four-element loss function is constructed based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image. The training data set for the ancient painting and calligraphy image reconstruction is trained in a supervised learning manner, a four-element loss function minimization is taken as a training target, and the ancient painting and calligraphy image reconstruction model is trained by continuously optimizing model parameters until a stop iteration condition is met.

[0013] Optionally, the detail enhancement feature map is processed by using a detail enhancement convolution module to obtain a detail enhancement value, including: A group of weights is calculated for each spatial position and each output channel of the detail enhancement feature map to obtain attention weights corresponding to different dilation rate hollow convolution branches; The detail enhancement feature map is subjected to parallel convolution operation by using a plurality of hollow convolution kernels with different dilation rates to obtain convolution results of different dilation rate hollow convolution branches; The convolution results of the different dilation rate hollow convolution branches are multiplied by the corresponding attention weights respectively, summed, and added to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0014] Optionally, a clear reconstructed painting and calligraphy image is generated based on the detail enhancement value, including: The detail enhancement map composed of the detail enhancement value is spliced with the preprocessed image along the channel dimension, and then input to an initial reconstruction subnetwork mapping function to obtain an initial reconstructed image; Based on the initial reconstructed image, an encoding model of an autoencoder is used for encoding operation and a decoding model is used for decoding operation, and a fully connected network is used to generate the clear reconstructed painting and calligraphy image.

[0015] An image reconstruction apparatus, comprising: An image acquisition unit configured to acquire an ancient painting and calligraphy image to be reconstructed; A preprocessing unit configured to perform multi-scale degradation simulation on the ancient painting and calligraphy image to be reconstructed, and perform adaptive degradation correction on the ancient painting and calligraphy image to be reconstructed by using the degradation simulation result to obtain a preprocessed image to be reconstructed; A feature enhancement unit configured to perform fuzzy detail feature enhancement processing on the preprocessed image to be reconstructed, and extract a target detail enhancement feature map from the preprocessed image to be reconstructed after the enhancement processing; An image reconstruction unit configured to input the target detail enhancement feature map to a pre-trained ancient painting and calligraphy image reconstruction model to perform painting and calligraphy image reconstruction processing to obtain a clear reconstructed ancient painting and calligraphy image, wherein the ancient painting and calligraphy image reconstruction model is trained by taking four-element loss function minimization as a training target and continuously optimizing model parameters until a stop iteration condition is met, and the four-element loss function is a loss function combining pixel fidelity loss, multi-scale perception loss, local stroke coherence loss and qi and rhythm consistency loss.

[0016] A computer storage medium stores at least one instruction, which is executed by a processor to implement the image reconstruction method described above.

[0017] An electronic device comprises a memory and a processor. The memory is used to store at least one instruction. The processor is used to execute the at least one instruction to implement the image reconstruction method described above.

[0018] From the above technical solution, the present application discloses an image reconstruction method and related device, obtaining an image to be reconstructed ancient painting, performing multi-scale degradation simulation on the image to be reconstructed ancient painting, and using the degradation simulation result to perform adaptive degradation correction on the image to be reconstructed ancient painting to obtain a preprocessed image to be reconstructed, performing fuzzy detail feature enhancement processing on the preprocessed image to be reconstructed, and extracting a target detail enhancement feature map from the preprocessed image to be reconstructed after enhancement, inputting the target detail enhancement feature map into a pre-trained ancient painting image reconstruction model for painting image reconstruction processing to obtain a reconstructed clear ancient painting image. The present application can effectively process the non-uniform degradation phenomenon of the image in space and scale, enhance the detail performance and color information of the image by performing multi-scale degradation simulation on the original ancient painting image and using the degradation simulation result to perform adaptive degradation correction on the image to be reconstructed ancient painting. By performing fuzzy detail feature enhancement processing on the preprocessed image to be reconstructed, the noise interference is effectively suppressed while the fuzzy details are enhanced, and the weak artistic details are enhanced on the basis of retaining the overall structure information of the image. By introducing a four-element loss function of comprehensive pixel fidelity loss, multi-scale perception loss, local brush stroke coherence loss and artistic style consistency loss during training of the ancient painting image reconstruction model, the image clarity is improved while the brushwork charm and aesthetic characteristics of the original work are maintained. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the disclosed drawings without creative labor.

[0020] Figure 1 A flowchart of an image reconstruction method disclosed in an embodiment of the present application; Figure 2 A training method flowchart of an ancient painting image reconstruction model disclosed in an embodiment of the present application; Figure 3 A quality comparison histogram of different methods in each feature extraction stage is disclosed in the embodiment of the present application; Figure 4 A curve graph of peak signal-to-noise ratio changing with scale number is disclosed in the embodiment of the present application; Figure 5 A curve graph of structural similarity changing with scale number is disclosed in the embodiment of the present application; Figure 6 A curve graph of edge preservation changing with scale number is disclosed in the embodiment of the present application; Figure 7 A curve graph of processing time changing with scale number is disclosed in the embodiment of the present application; Figure 8 A structural schematic diagram of an image reconstruction device is disclosed in the embodiment of the present application; Figure 9 A structural schematic diagram of an electronic device is disclosed in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0022] The embodiments of the present application disclose an image reconstruction method and related device, by performing multi-scale degradation simulation on original ancient calligraphy and painting images, and using the degradation simulation results to perform adaptive degradation correction on the to-be-reconstructed ancient calligraphy and painting images, the non-uniform degradation phenomenon of the images in space and scale can be effectively processed, and the detail performance and color information of the images are enhanced; by performing fuzzy detail feature enhancement processing on the preprocessed to-be-reconstructed images, the fuzzy details are targetedly enhanced while the noise interference is effectively suppressed, the weak artistic details are enhanced on the basis of retaining the overall structural information of the images; by introducing a four-element loss function of comprehensive pixel fidelity loss, multi-scale perception loss, local brush stroke coherence loss and artistic style consistency loss during training of the ancient calligraphy and painting image reconstruction model, the image clarity is improved while the brush and ink charm and aesthetic characteristics of the original work are maintained.

[0023] Referring to Figure 1 The embodiment of the present application discloses an image reconstruction method flow chart, and the method comprises the following steps: Step S101, obtaining a to-be-reconstructed ancient calligraphy and painting image.

[0024] The to-be-reconstructed ancient calligraphy and painting image refers to a new, untreated ancient calligraphy and painting image.

[0025] In step S102, multi-scale degradation simulation is performed on the to-be-reconstructed ancient calligraphy and painting image, and adaptive degradation correction is performed on the to-be-reconstructed ancient calligraphy and painting image by using the degradation simulation result, to obtain a preprocessed to-be-reconstructed image.

[0026] The ancient calligraphy and painting image usually has degradation phenomena such as blurring, detail loss, and color fading due to long time, and such degradation has multi-scale and spatial non-uniformity, that is, the degradation degree and mode of different regions at different scales are different. Conventional techniques such as global normalization or histogram equalization cannot effectively process such complex non-uniform degradation because they assume that the degradation process is uniform in the global range of the image, and often lead to over-processing of detail regions or insufficient processing of smooth regions, which cannot provide high-quality preprocessed basic data for subsequent reconstruction.

[0027] The present application can effectively process the non-uniform degradation of the image in space and scale by performing multi-scale degradation simulation on the original ancient calligraphy and painting image and performing adaptive degradation correction on the to-be-reconstructed ancient calligraphy and painting image by using the degradation simulation result, and can enhance the detail performance and color information of the image to provide strong support for subsequent feature extraction.

[0028] In step S103, fuzzy detail feature enhancement processing is performed on the preprocessed to-be-reconstructed image, and a target detail enhancement feature map is extracted from the preprocessed to-be-reconstructed image after enhancement processing.

[0029] The fuzzy details of the ancient calligraphy and painting image are often hidden in the low-frequency components or textures of the image, and the edge and texture signals are weak. Conventional edge detection operators or high-frequency filtering cannot effectively distinguish between real details and noise when processing such images, and are prone to amplify noise introduced in the image degradation process, resulting in the details being submerged and unable to provide effective feature representation for subsequent reconstruction.

[0030] Therefore, the present application can effectively suppress noise interference while enhancing the fuzzy details by performing fuzzy detail feature enhancement processing on the preprocessed to-be-reconstructed image, improve the recognizability of weak details on the basis of retaining the overall structural information of the image, and lay a solid foundation for accurate reconstruction of ancient calligraphy and painting images, replacing the edge detection or high-frequency filtering operation of conventional methods, thereby effectively solving the problem of destroying the original artistic strokes in conventional methods.

[0031] In step S104, the target detail enhancement feature map is input into a pre-trained ancient calligraphy and painting image reconstruction model for calligraphy and painting image reconstruction processing, to obtain a reconstructed clear ancient calligraphy and painting image.

[0032] The ancient calligraphy and painting image reconstruction model is: the training data set of ancient calligraphy and painting image reconstruction is trained in a supervised learning manner, a four-element loss function is minimized as a training target, and the model parameters are continuously optimized to meet the stop iteration condition to obtain the training result.

[0033] It should be noted that the four-element loss function in the present application is a loss function combining pixel fidelity loss, multi-scale perception loss, local stroke coherence loss and qi rhythm consistency loss.

[0034] The pixel fidelity loss can ensure that the reconstructed ancient calligraphy and painting image is as close as possible to the original image at the pixel level.

[0035] The multi-scale perception loss evaluates and optimizes the image from different scales. On a macro scale, the overall composition and layout of the reconstructed image are consistent with the original calligraphy and painting; on a micro scale, attention is paid to the presentation of details such as stroke thickness and ink color variation. This multi-scale consideration makes the reconstructed image highly similar to the original work at all levels, improving the overall quality of the image.

[0036] The strokes of ancient calligraphy and painting are an important embodiment of its artistic style, and each stroke of the brush, the line and the end of the brush all contain the artist's emotions and skills. The local stroke coherence loss ensures the smoothness and consistency of the strokes in the reconstruction process, avoiding problems such as broken strokes and incoherence. For example, in the reconstruction of calligraphy works, the turning of the strokes is natural and the force is uniform, so that the reconstructed calligraphy work still has a sense of rhythm and rhythm.

[0037] The qi rhythm of calligraphy and painting is the soul of ancient calligraphy and painting, which embodies the overall artistic conception and artistic appeal of the work. The four-element loss function takes into account the qi rhythm consistency loss, analyzes the qi rhythm characteristics of the original work, such as the smoothness of lines, the harmony of colors, and the appropriate density of composition, and tries to preserve and inherit these qi rhythms in the reconstruction process. The reconstructed ancient calligraphy and painting not only looks similar, but also has the same artistic connotation as the original work, allowing the audience to feel the unique charm of ancient calligraphy and painting.

[0038] Therefore, the four-element loss function provides a comprehensive evaluation index for the training of the ancient calligraphy and painting reconstruction model. It considers multiple aspects of the image, allowing the model to optimize from different angles during training, avoiding the local optimum problem that may be caused by a single loss function. By minimizing the four-element loss function, the model can learn more comprehensive and accurate image features, thereby improving the accuracy and quality of the reconstruction. It effectively breaks through the limitations of traditional U-Net network structure and loss function, which only focuses on pixel accuracy, and realizes the collaborative reconstruction of artistic details and style characteristics of calligraphy and painting images.

[0039] In summary, the application discloses an image reconstruction method, obtains an ancient calligraphy and painting image to be reconstructed, performs multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and utilizes the degradation simulation result to perform self-adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed to obtain a pretreated image to be reconstructed, performs fuzzy detail feature enhancement processing on the pretreated image to be reconstructed, and extracts a target detail enhancement feature map from the pretreated image to be reconstructed after enhancement processing, inputs the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, and obtains a reconstructed clear ancient calligraphy and painting image. The application can effectively process non-uniform degradation phenomena of an image in space and scale, enhance detail performance and color information of the image, and effectively suppress noise interference while performing targeted enhancement on fuzzy details on the basis of retaining overall structural information of the image, by performing multi-scale degradation simulation on an original ancient calligraphy and painting image and utilizing the degradation simulation result to perform self-adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed. The four loss functions of comprehensive pixel fidelity loss, multi-scale perception loss, local brush stroke coherence loss and artistic style consistency loss are introduced during training of the ancient calligraphy and painting image reconstruction model, so that the artistic style and aesthetic characteristics of the original work can be maintained while improving image clarity.

[0040] In one embodiment, step S102 can specifically include: (1) For the ancient calligraphy and painting image to be reconstructed, a convolution kernel representing local blur or degradation effect of a spatial position is estimated at each spatial position and a preset scale, and is taken as a local degradation kernel. Wherein, the estimation of the local degradation kernel is realized by minimizing the difference between a local image block with the current position as the center and a corresponding scale and an ideal clear template. The ideal clear template can be obtained from high-quality contemporary calligraphy and painting data or generated by degradation prior knowledge.

[0041] (2) The local degradation kernel is utilized to perform degradation simulation on the ancient calligraphy and painting image to be reconstructed at different scales. (3) The simulation results at different scales are adaptively fused according to the detail richness degree of each position and local region, and a bias term is added to adjust the color for self-adaptive degradation correction, so as to obtain the pretreated image to be reconstructed.

[0042] In one embodiment, step S103 can specifically include: (1) For each color channel of the pretreated image to be reconstructed, a target multi-directional gradient amplitude is calculated. In practical applications, for each color channel of the preprocessed image, the gradient amplitude value of the color channel is calculated along four preset directions, and the gradient amplitude value represents edge and texture change information of the preprocessed image in different directions.

[0043] (2) Based on the absolute deviation of the current pixel value of the preprocessed image to be reconstructed and the global average value of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated. To adaptively fuse the gradient information of the three color channels, an adaptive weight is calculated for each spatial position, each output feature channel and each input color channel, and the adaptive weight is calculated based on the absolute deviation of the current pixel value and the global average value of the color channel to which the current pixel value belongs. The channel with a larger deviation obtains a higher weight.

[0044] (3) A target local enhancement factor is calculated according to the texture complexity and detail richness of the local region. A spatially adaptive local enhancement factor is calculated, and the local enhancement factor can dynamically adjust the gradient enhancement strength according to the texture richness of the local region: stronger enhancement effect is applied in the texture complex region, and the enhancement strength is weakened in the flat region, so as to avoid the noise being amplified.

[0045] (4) The target multi-directional gradient amplitude value, the target channel fusion weight and the target local enhancement factor are combined, a nonlinear activation operation is performed, and a bias term is added to generate the target detail enhancement feature map.

[0046] In one embodiment, referring to Figure 2 The training method flowchart of the ancient painting and calligraphy image reconstruction model disclosed by the embodiment of the present application comprises the following steps: Step S201, constructing a training data set for ancient painting and calligraphy image reconstruction.

[0047] Each sample in the training data set is an ancient painting and calligraphy image pair, and the ancient painting and calligraphy image pair comprises an original ancient painting and calligraphy image with different degrees of degradation and a high-quality clear reference image corresponding to the original ancient painting and calligraphy image.

[0048] In each sample collection process, first, for the selected ancient painting and calligraphy cultural relics, a high-resolution professional scanner or a professional digital camera equipped with a specific cultural asset shooting lens is used to collect digital images under strict control of light conditions, and an initial high-definition digital image is obtained as a high-quality clear reference image.

[0049] In order to simulate the complex degradation of calligraphy and painting in the real world due to long time, so as to construct the ancient calligraphy and painting image pair as a sample, the application carries out artificial degradation processing on the high-quality clear reference image. The degradation processing process strictly simulates the multi-scale and non-uniform characteristics, including but not limited to simulating ink bleeding, blur caused by paper fiber aging, uneven color fading in different areas, and spot or block stains, mold spots and the like, so as to obtain the original ancient calligraphy and painting image corresponding to the high-quality clear reference image.

[0050] Among them, the original ancient calligraphy and painting image and the high-quality clear reference image are both in RGB (Red-Green-Blue) format.

[0051] Step S202, multi-scale degradation simulation is carried out on the original ancient calligraphy and painting image in each sample of the training data set, and self-adaptive degradation correction is carried out on the original ancient calligraphy and painting image by using the degradation simulation result, to obtain a preprocessed image.

[0052] The application carries out multi-scale degradation simulation on the original ancient calligraphy and painting image, and adaptively estimates the local degradation kernel of the original ancient calligraphy and painting image at different spatial positions and scale levels. Then, the estimated local degradation kernel is used to implement weighted fusion preprocessing on the original ancient calligraphy and painting image, to obtain a preprocessed image. This process can effectively simulate and preliminarily correct the non-uniform degradation phenomenon of the image, enhance the detail performance and color information of the image, and provide strong support for subsequent feature extraction work.

[0053] Step S203, blur detail feature enhancement processing is carried out on the preprocessed image, and a detail enhancement feature map is extracted from the preprocessed image after enhancement processing.

[0054] The blurred details of the ancient calligraphy and painting image are often hidden in the low-frequency components or textures of the image, and the edge and texture signals are weak. The conventional edge detection operator or high-frequency filter is difficult to effectively distinguish the true details from the noise when processing such images, and is also easy to amplify the noise introduced in the image degradation process, so that the detail features are submerged and cannot provide effective feature representation for subsequent reconstruction.

[0055] Therefore, the application carries out blur detail feature enhancement processing on the preprocessed image by using an adaptive feature enhancement function, combines local contrast adjustment and multi-directional gradient information fusion, extracts and enhances the blurred detail features from the preprocessed image, and the function can dynamically adjust the enhancement intensity according to the texture complexity of the local area, and adaptively fuse the gradient information of different color channels, to generate a robust and detail-rich detail enhancement feature map.

[0056] Step S204, based on the detail enhancement feature map, training the training data set in a supervised learning manner to obtain the ancient painting and calligraphy image reconstruction model.

[0057] In one embodiment, step S202 can specifically include: 1) For each of the original ancient painting and calligraphy images, estimate a convolution kernel representing the local blur or degradation effect of the spatial position at each spatial position and a preset scale, and take it as a local degradation kernel.

[0058] Wherein, the estimation of the local degradation kernel is realized by minimizing the difference between the local image block centered at the current position and corresponding scale and the ideal clear template. Wherein, the ideal clear template can be obtained from high-quality contemporary painting and calligraphy data, or can be generated by degradation priori knowledge. The expression of the local degradation kernel is shown in formula (1): (1); In the formula, represents the local degradation kernel estimated at the spatial position and the s-th scale, which is used to represent the blur or degradation effect of the corresponding local area at a specific scale; x represents the row index in the image, which is used to locate the vertical position of the pixel; y represents the column index in the image, which is used to locate the horizontal position of the pixel; s represents the scale index, which is used to specify the observation scale of the degradation modeling, and the value range is ; S represents the preset total number of scales, which is used to control the granularity of multi-scale analysis; represents the original ancient painting and calligraphy image in RGB format; represents the convolution kernel K obtained by optimization solution to minimize the subsequent loss function; represents the operation of extracting the local image block defined by the position as the center and the s-th scale from the image; represents the two-dimensional convolution operator; represents the ideal clear image block template at the position and the s-th scale; represents the two-norm operator of the vector.

[0059] ​​It should be noted that the value of each pixel point of the original ancient painting and calligraphy image represents the light intensity or reflectivity of the position in the red, green and blue color channels, wherein is the input data of the preprocessing procedure.

[0060] It should be noted that the total scale number S is set according to the size range of the typical degradation features in the painting and calligraphy image, for example, it can be 3, to model the degradation effects in different scales such as small brush strokes, medium texture and large faded areas.

[0061] 2) Degradation simulation of the original ancient painting and calligraphy image in different scales is performed using the local degradation kernel, and the simulation results in different scales are adaptively fused according to the detail richness of the local area at each position, while a bias term is added to adjust the color for adaptive degradation correction, to obtain a preprocessed image.

[0062] wherein the calculation expression of the preprocessed image is shown in formula (2): (2); In the formula, represents the preprocessed image The pixel value of the cth color channel at position ; c represents the color channel index, and the value range is , respectively corresponding to the red channel, the green channel and the blue channel; represents the summation of all preset scales s; represents the adaptive fusion weight corresponding to the sth scale at position , that is, the scale weight; represents that the degradation kernel is used to locally convolve the cth color channel of the original ancient painting and calligraphy image to simulate the degradation effect at this scale; represents the learnable bias term at position , the cth color channel, which is used to correct the color deviation or brightness deviation that may occur after image preprocessing.

[0063] wherein the scale weight is calculated based on the variance of the corresponding local image block. The area with larger variance is usually assigned a higher weight to retain more details, and its calculation method is shown in formula (3): (3); In the formula, represents the natural exponential function; This represents the scale-sensitive parameter corresponding to the s-th scale. It is an adjustable hyperparameter used to control the effect of local variance on the scale weights. The intensity of the impact; This represents the variance operator, used to calculate the variance of pixel values ​​in a local image patch; Indicated by position A local image patch defined at the s-th scale, centered on ; The dummy index in the summation formula represents the scale and is used to calculate the normalized denominator for all weights. Indicates corresponding to the first The scale-sensitive parameters at each scale are adjustable hyperparameters used to control the effect of local variance on scale weights. The intensity of the impact; Indicated by position Centered on, the A local image patch defined by a scale.

[0064] In one embodiment, step S203 may specifically include: (1) For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude.

[0065] In practical applications, for each color channel of the preprocessed image, its gradient magnitude is calculated along four preset directions. This gradient magnitude represents the edge and texture variation information of the preprocessed image in different directions, specifically expressed as follows: (4); In the formula, Indicates the location of the preprocessed image. The c-th color channel, direction The gradient magnitude is used to characterize the rate of intensity change of the preprocessed image at that location along a specific direction; Represents the gradient direction, with values ​​from a set. These represent horizontal, 45-degree diagonal, vertical, and 135-degree diagonal directions, respectively. Indicates the direction Gradient operators, such as the Sobel operator.

[0066] In practice, the gradient magnitude calculation in the four directions is completed through convolution operations, and a specific gradient kernel is used to filter the image in each direction.

[0067] (2) Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel.

[0068] To adaptively fuse the gradient information of the three color channels, adaptive weights are calculated for each spatial location, each output feature channel, and each input color channel. These adaptive weights are calculated based on the absolute deviation of the current pixel value from the global mean of its corresponding color channel. Channels with larger deviations receive higher weights. The calculation method is shown in formula (5). (5); In the formula, Indicates the location At this point, an adaptive weight is used to fuse the gradient information of the input color channel c into the output feature channel d. The weight value is between 0 and 1. d represents the channel index of the output detail feature map, and its value ranges from 0 to 1. ; Indicates the total number of feature channels; This represents a learnable weight adjustment parameter used to control the strength of the influence of pixel value differences in the input color channel c on the weights fused into the feature channel d. This represents the absolute value operator; This represents the global pixel mean of the c-th color channel of the preprocessed image across the entire dataset, used for centering. Indicates the location of the preprocessed image. The pixel value on the c-th color channel; This represents the dummy index in the summation formula, which represents the color channel and is used to calculate the normalized denominator for the weights. Indicates the location of the preprocessed image. , No. Pixel values ​​on each color channel; This indicates that the preprocessed image is on the entire dataset, the first... The global pixel mean of each color channel is used for centering.

[0069] (3) Calculate the local enhancement factor based on the texture complexity and detail richness of the local region.

[0070] The computation space is adapted to a local enhancement factor that dynamically adjusts the gradient enhancement intensity based on the texture richness of the local region: applying a stronger enhancement effect to regions with complex textures and a weaker enhancement effect to regions with flat textures, thereby avoiding the amplification of noise.

[0071] The expression for the local enhancement factor is as follows: (6); In the formula, Indicates the location The local enhancement factor at that location is greater than or equal to scalar; This represents the global enhancement intensity parameter, which is an adjustable hyperparameter used to control the overall enhancement magnitude; Represent the natural logarithm function; Indicated by position The standard deviation of pixel values ​​within a local neighborhood centered on the pixel is used to measure the texture complexity and detail richness of that local region.

[0072] (4) Combine the multi-directional gradient magnitude, the channel fusion weight and the local enhancement factor, and generate a detail enhancement feature map by non-linear activation operation and adding a bias term.

[0073] The expression for the detail-enhancing feature map is as follows: (7); In the formula, This indicates the location of the output detail-enhanced feature map. The value on the feature channel d; This represents the modified linear unit activation function, used to introduce nonlinearity and filter out possible negative responses; This represents the set of four preset gradient directions. Perform summation; This represents the learnable spatial bias term corresponding to feature channel d, used to adjust the baseline of the feature map.

[0074] In one embodiment, step S204 may specifically include: (1) The detail enhancement feature map is processed using the detail enhancement convolution module to obtain the detail enhancement value.

[0075] Conventional convolutional neural networks, due to their fixed receptive field and single convolution mode, may not be able to effectively capture multi-scale details hidden in images when processing blurry details in ancient paintings and calligraphy. This results in some scale details being ignored or smoothed out during propagation.

[0076] Based on this, the present invention constructs a detail enhancement convolution module, which captures multi-scale contextual information of detail enhancement feature maps through parallel dilated convolution with multiple dilation rates, and introduces a spatially adaptive attention mechanism to dynamically fuse feature responses at different scales, thereby enhancing the network's ability to perceive blurred and scale-variable details in calligraphy and painting.

[0077] The process of obtaining detail enhancement values ​​using the detail enhancement convolution module specifically includes: 1) Calculate a set of weights for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates.

[0078] For each spatial location and each output channel, corresponding attention weights are calculated. These weights measure the importance of different dilation rates of the dilated convolutional branches for the specific features of that spatial location and output channel. Specifically, the attention score for each dilated convolutional branch is first calculated, and then these scores are normalized using the Softmax function, as follows: (8); In the formula, This represents the attention weights of the dilation-rate dilated convolution branch, i.e., at position. , corresponding to the normalized attention weights of the r-th expansion rate and the o-th output channel; r represents the expansion rate index, and its value range is... ; R represents the total preset expansion rate; o represents the module output channel index, with a value range of 1. ; Indicates the total number of output channels of the module; Indicates the location The attention score corresponds to the r-th expansion rate and the o-th output channel. The higher the score, the greater the contribution of the scale branch at that position to the formation of the o-th output channel feature. Let represent a set of learnable parameters used to generate attention scores, which establishes the spatial correlation between the input feature channel d and the output channel o at the r-th expansion rate; This represents the dummy index in the summation formula, which represents the expansion rate and is used to calculate the normalized denominator of the weights. Let represent a set of learnable parameters used to generate attention scores, which establishes the relationship between the input feature channel d and the output channel o at the th... Spatial correlation under an expansion rate.

[0079] Among them, attention score The calculation formula is as follows: (9); In the formula, This represents summing over all output channels of the input detail-enhancing feature map; Let represent a set of learnable parameters used to generate attention scores, which establishes the spatial correlation between the input feature channel d and the output channel o at the r-th expansion rate; This indicates the location of the output detail-enhanced feature map. The value on the feature channel d.

[0080] 2) Perform parallel convolution operations on the detail enhancement feature map using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates.

[0081] Multiple dilated convolutional kernels with different dilation rates are used to perform convolution operations in parallel on the input detail enhancement feature map. These kernels share the same spatial size but have different dilation rates, thus enabling the acquisition of receptive fields of different scales without increasing the number of parameters. This operation outputs a three-dimensional tensor, specifically represented as: (10); In the formula, The dilated convolution branch with the r-th dilation rate is located at position r. The convolution result on the o-th output channel; This represents a dilated convolution operation with an expansion rate of r, which expands the receptive field by inserting zero values ​​between the weights of a standard convolution kernel. The weight parameter corresponding to the o-th output channel in the dilated convolution kernel with the r-th dilation rate; This indicates the location of the input detail-enhanced feature map. All channel values ​​at that location.

[0082] 3) Multiply the convolution results of the dilated convolution branches with the corresponding attention weights respectively, sum them, and add the summation result to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0083] (11); In the formula, This indicates that the detail-enhancing convolutional module is located at... The detail enhancement value of the o-th output channel; This represents summing the results of all R dilated convolution branches with different dilation rates; This represents the learnable bias parameter corresponding to the o-th output channel, used to adjust the baseline of the output characteristics.

[0084] (2) Based on the detail enhancement value output by the detail enhancement convolution module, a reconstructed clear calligraphy and painting image is generated.

[0085] Conventional image reconstruction networks such as U-Net and its variants typically use mean squared error or L1 loss function for supervised training. These loss functions focus on pixel-level average accuracy, but they can easily lead to overly smooth reconstruction results, losing the sharpness of brushstrokes and the layering of ink tones unique to calligraphy and painting images, making it difficult to meet the strict requirements for restoring the characteristics of calligraphy and painting art.

[0086] This invention constructs a multi-stage cyclic refinement reconstruction network and introduces a quaternary loss function. This multi-stage cyclic refinement reconstruction network first deeply fuses the detail enhancement values ​​output by the detail enhancement convolution module with the preprocessed image to generate an initial reconstructed image. Then, iterative detail refinement is performed on the initial result through a cyclic feedback structure. The introduced quaternary loss function not only considers pixel-level accuracy but also innovatively introduces a consistency loss based on multi-scale statistical features of the original preprocessed image, forcing the reconstructed result to maintain consistency with the original calligraphy and painting in terms of texture style and overall visual perception. The specific steps are as follows: 1) The detail enhancement map, which is composed of the detail enhancement values ​​output by the detail enhancement convolution module, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstruction image.

[0087] The initial reconstructed image, which integrates low-level color information and high-level multi-scale detail features, is represented as follows: (12); In the formula, This represents the RGB pixel vector at row u and column v of the final reconstructed clear image of the calligraphy and painting. u represents the row index in the initial reconstructed image space, used to locate the vertical position of the pixel; v represents the column index in the initial reconstructed image space, used to locate the horizontal position of the pixel; The parameter is The initial reconstruction subnetwork mapping function consists of convolutional layers, nonlinear activation functions, and upsampling operations. This indicates a splicing operation along the channel dimension; This represents the vector consisting of all output feature channel values ​​at row u and column v of the detail enhancement convolution module; Indicates preprocessed image A vector consisting of the RGB three-channel values ​​at the u-th row and v-th column; This represents the set of all trainable parameters, such as weights and biases, in the initial reconstructed subnetwork.

[0088] In the specific implementation, the range of values ​​for the row index u and column index v is consistent with the size of the clearly reconstructed ancient calligraphy and painting image. If the clearly reconstructed ancient calligraphy and painting image is H×W pixels, then... , .

[0089] 2) Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and a fully connected network is combined to generate a reconstructed clear calligraphy and painting image.

[0090] The entire process is represented as follows: (13); In the formula, Reconstructing clear images of calligraphy and paintings The RGB pixel vector at row u and column v; The parameter is The fully connected network is specifically implemented using two fully connected layers; This represents the encoding model of an autoencoder; This represents the decoding model of an autoencoder; This represents the set of trainable parameters for a fully connected network.

[0091] (3) Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed.

[0092] The expression for the quaternary loss function is as follows: (14); In the formula, The quaternion loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local stroke coherence loss, and rhythm consistency loss. This represents the balancing weight of the pixel loss term, with an example value of 1.0. This represents the balancing weight of the perceptual loss term, with an example value of 0.1. This represents the balancing weight of the stroke loss term, with an example value of 0.5. This represents the balancing weight of the loss of spirit and charm, with an example value of 0.2. This represents the pixel fidelity loss, specifically employing a smoothed L1 loss to enhance robustness against outliers. Its calculation method is as follows: (15); In the formula, N represents the total number of pixels in a single image; c represents the color channel index, with a value range of {1, 2, 3}, corresponding to the red, green, and blue channels respectively; Representing authentic and clear images of ancient calligraphy and paintings. In the line, number Column, No. The pixel values ​​of each channel; This represents the smoothed L1 loss function.

[0093] The multi-scale perceptual loss is represented by comparing the feature differences between the reconstructed image and the real image at multiple layers of the pre-trained VGG network. Its calculation method is expressed as follows: (16); In the formula, I represents the selected feature layer index; L represents the set of selected feature layers, for example {'relu1_2', 'relu2_2', 'relu3_3', 'relu4_3'}; This indicates the pre-trained VGG network's... Feature extraction function of the layer; This indicates the pre-trained VGG network's... Layer-by-layer reconstruction of clear calligraphy and painting images Feature extraction function; This indicates the pre-trained VGG network's... Layers of realistic and clear images of ancient calligraphy and paintings Feature extraction function; Indicates the first The height of the layer feature map; Indicates the first Width of the layer feature map; Indicates the first The number of channels in the layer feature map.

[0094] This represents the loss of local stroke coherence, emphasizing the consistency between the reconstructed image and the real image in the distribution of local gradient magnitudes, in order to maintain the sharpness of stroke edges. Its calculation method is expressed as follows: (17); In the formula, Reconstructing clear images of calligraphy and paintings In the line, number Gradient magnitude at column; Representing authentic and clear images of ancient calligraphy and paintings. In the line, number The gradient magnitude at the column.

[0095] To represent the loss of stylistic consistency, and to ensure that the reconstructed image remains faithful to the original in terms of overall texture style statistics, the Gram matrix difference between the reconstructed image and the upsampled original preprocessed image on multi-scale features is calculated. The calculation method is as follows: (18); In the formula, This represents the set of feature layers used for style comparison, for example, {'relu2_2', 'relu3_3', 'relu4_3'}; This indicates a bicubic interpolation upsampling operation, which scales the image to match the reconstructed clear image of the calligraphy or painting. Same size; This indicates that for dimension 1 The Gram matrix calculated from the F1 feature map of the l-th layer; p represents the row index of the Gram matrix, corresponding to the feature channel, and its value range is... ; This represents the number of channels in the feature map of the l-th layer; This indicates the pre-trained VGG network's... Layer to the first Layer-by-layer reconstruction of clear calligraphy and painting images Gram matrix calculated from features extracted from authentic and clear ancient calligraphy and painting images; This indicates the pre-trained VGG network's... Layer-by-layer preprocessed image Gram matrix calculated from features extracted by bicubic interpolation upsampling; This represents the Frobenius norm of the matrix.

[0096] In one embodiment, the feature quality of different methods at different feature extraction stages is compared. The experiment compares the image reconstruction method proposed in this invention with four conventional techniques: global normalization, histogram equalization, a super-resolution reconstruction method based on convolutional neural networks, and the U-Net deep learning reconstruction network. The experiment scores the image reconstruction of ancient calligraphy and painting images from four feature dimensions crucial to their reconstruction: edge preservation (assessing the sharpness of brushstroke contours), texture sharpness (assessing the clarity of paper fibers and ink texture), color fidelity (assessing the accuracy of color reproduction), and detail richness (assessing the degree of preservation of subtle brushstrokes and faded area details). The experiment uses the same test dataset, and professional researchers blindly score the reconstruction results to ensure objectivity. Figure 3 The bar chart showing the quality comparison of different methods at each feature extraction stage reveals that the image reconstruction method disclosed in this invention achieved the highest scores across all four feature dimensions, with particularly significant advantages in detail richness and color fidelity. The two traditional methods (global normalization and histogram equalization) scored lower across all features because their globally uniform processing cannot adapt to the non-uniform degradation characteristics of ancient calligraphy and paintings. The two deep learning methods performed reasonably well on some features, but were significantly lacking in detail richness. Experimental results demonstrate that the image reconstruction method disclosed in this invention, through its adaptive detail enhancement function and multi-scale attention mechanism, can more comprehensively and accurately restore and enhance various artistic features of ancient calligraphy and paintings.

[0097] In one embodiment, the performance of the image reconstruction method disclosed in this invention is analyzed at different scale numbers, investigating the impact of the key hyperparameter—scale number—of the multi-scale degradation modeling module on the overall performance. In the experimental setup, all other parameters were fixed, and only the scale number was systematically changed. Four key metrics were evaluated: peak signal-to-noise ratio (PSNR) (an objective indicator of reconstruction quality), structural similarity (a perceptual quality indicator), edge preservation (a brushstroke detail indicator), and processing time (an efficiency indicator). The units for the first three metrics are decibels, dimensionless, and dimensionless, respectively; higher values ​​indicate better performance. The unit for processing time is seconds.

[0098] Figure 4 The graph shows the peak signal-to-noise ratio as a function of scale number. Peak signal-to-noise ratio (PSNR) is an objective indicator of reconstruction quality. Figure 5 The graph shows the variation of structural similarity with scale number; Figure 6 The graph shows the edge preservation degree as a function of scale number; Figure 7 The graph shows the processing time as a function of the number of scales.

[0099] from Figures 4-7 As can be seen, with the increase of the number of scales, the peak signal-to-noise ratio, structural similarity, and edge preservation all rise rapidly at first, approaching their peak at a scale of 4 or 5, and then slowly decline or tend to plateau. Processing time, however, increases approximately linearly with the number of scales. Experimental results show that a larger scale is not always better: too few scales prevent the model from fully capturing multi-scale degradation features, leading to poor performance; too many scales, while potentially capturing finer-grained features, significantly increase computational complexity and may introduce redundancy or noise, resulting in limited or even decreased performance improvement. Therefore, the existence of an optimal scale range provides a clear experimental basis for the selection of the scale for multi-scale modeling in this invention, ensuring the best balance between effectiveness and efficiency.

[0100] (4) For the training dataset for the reconstruction of ancient calligraphy and painting images, a supervised learning method is adopted for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

[0101] In this study, supervised learning was used to update all training parameters in the ancient calligraphy and painting image reconstruction model.

[0102] Specifically, in each iteration, original ancient calligraphy and painting images with varying degrees of degradation from the training dataset are input into the ancient calligraphy and painting image reconstruction model to obtain the reconstructed clear calligraphy and painting images output by the model. A quaternary loss function is calculated based on the reconstructed clear calligraphy and painting images and the corresponding high-quality clear reference images from the training dataset. Based on the calculation results of the quaternary loss function, the gradient of each learnable parameter in the ancient calligraphy and painting image reconstruction model is adjusted using the backpropagation algorithm. This process is iterated continuously until the quaternary loss function is minimized.

[0103] After each training cycle, the quaternary loss function value is calculated on the validation set using the current model parameters. The stopping condition is set based on the validation set performance, specifically employing an "early stopping" strategy. During this strategy, the quaternary loss function result calculated on the validation set is continuously monitored. If the quaternary loss function value does not decrease within five consecutive training cycles, the model performance is considered to have reached saturation or overfitting has begun. At this point, the stopping condition is triggered, terminating the training cycle. Finally, the ancient calligraphy and painting image reconstruction model from the last iteration is selected as the completed ancient calligraphy and painting image reconstruction model.

[0104] Corresponding to the above method embodiments, the present invention discloses an image reconstruction apparatus.

[0105] See Figure 8 The present invention discloses a schematic diagram of an image reconstruction apparatus, which may include: Image acquisition unit 301 is used to acquire images of ancient paintings and calligraphy to be reconstructed.

[0106] Among them, the ancient calligraphy and painting images to be reconstructed refer to new, unprocessed ancient calligraphy and painting images.

[0107] The preprocessing unit 302 is used to perform multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results, so as to obtain the preprocessed image to be reconstructed.

[0108] Ancient paintings and calligraphy images often exhibit degradation phenomena such as blurring, loss of detail, and color fading due to their age. This type of degradation is multi-scale and spatially non-uniform, meaning that the degree and pattern of degradation vary in different regions at different scales. Conventional techniques such as global normalization or histogram equalization, which assume that the degradation process is uniform across the entire image, are difficult to effectively handle this complex non-uniform degradation. This often leads to over-processing of detailed areas or under-processing of smooth areas, failing to provide high-quality preprocessing data for subsequent reconstruction stages.

[0109] This invention performs multi-scale degradation simulation on original ancient calligraphy and painting images, and uses the degradation simulation results to perform adaptive degradation correction on the ancient calligraphy and painting images to be reconstructed. This can effectively handle the non-uniform degradation phenomenon of images in space and scale, enhance the detail and color information of the images, and provide strong support for subsequent feature extraction work.

[0110] The feature enhancement unit 303 is used to perform blurred detail feature enhancement processing on the preprocessed image to be reconstructed, and extract the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed.

[0111] The blurred details in ancient calligraphy and painting images are often hidden in the low-frequency components or textures of the image. The edge and texture signals are relatively weak. When processing such images, conventional edge detection operators or high-frequency filters not only have difficulty in effectively distinguishing between real details and noise, but also tend to amplify the noise introduced during the image degradation process, causing the detailed features to be submerged and unable to provide effective feature representation for subsequent reconstruction.

[0112] Based on this, the present invention enhances the blurred details of the preprocessed image to be reconstructed by performing blurred detail feature enhancement processing. This can effectively suppress noise interference while enhancing blurred details in a targeted manner. While preserving the overall structural information of the image, it improves the recognizability of weak details, thus laying a solid foundation for the accurate reconstruction of ancient calligraphy and painting images. This replaces the edge detection or high-frequency filtering operations of conventional methods, thereby effectively solving the problem of destroying the original artistic brushstrokes in conventional methods.

[0113] Image reconstruction unit 304 is used to input the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, and obtain a reconstructed clear ancient calligraphy and painting image.

[0114] The ancient calligraphy and painting image reconstruction model is as follows: the training dataset for ancient calligraphy and painting image reconstruction is trained using a supervised learning method, with the minimization of the quaternary loss function as the training objective. The model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

[0115] It should be noted that the quaternary loss function in this invention is a loss function that combines pixel fidelity loss, multi-scale perception loss, local stroke coherence loss, and rhythm consistency loss.

[0116] Among them, pixel fidelity loss can ensure that the reconstructed ancient calligraphy and painting images are as close as possible to the original images at the pixel level.

[0117] Multi-scale perceptual loss evaluates and optimizes images at different scales. At the macro scale, it ensures that the overall composition and layout of the reconstructed image are consistent with the original painting or calligraphy; at the micro scale, it focuses on the presentation of details, such as the thickness of brushstrokes and the variation in ink density. This multi-scale consideration ensures that the reconstructed image is highly similar to the original work at all levels, thus improving the overall image quality.

[0118] The brushstrokes in ancient Chinese calligraphy and painting are a crucial manifestation of their artistic style. Each stroke, from its beginning to its execution, embodies the artist's emotions and skill. The loss of continuity in local brushstrokes ensures the smoothness and consistency of the strokes during reconstruction, avoiding issues such as broken or disjointed strokes. For example, in reconstructing calligraphy, this ensures natural transitions and even pressure in the strokes, allowing the reconstructed work to retain its rhythm and flow.

[0119] The spirit and rhythm of calligraphy and painting are the soul of ancient Chinese art, embodying the overall artistic conception and appeal of the work. The quaternary loss function incorporates the loss of spirit and rhythm consistency, analyzing the spirit and rhythm characteristics of the original work, such as the fluidity of lines, the harmony of colors, and the appropriate density of composition, to preserve and inherit these spirit and rhythm as much as possible during the reconstruction process. This ensures that the reconstructed ancient calligraphy and painting not only resembles the original in appearance but also inherits its artistic essence, allowing viewers to experience the unique charm inherent in ancient calligraphy and painting.

[0120] Therefore, the quaternary loss function provides a comprehensive evaluation metric for training ancient calligraphy and painting reconstruction models. It comprehensively considers multiple aspects of the image, enabling the model to optimize from different angles during training and avoiding the local optima problem that may occur with a single loss function. By minimizing the quaternary loss function, the model can learn more comprehensive and accurate image features, thereby improving the accuracy and quality of reconstruction. It effectively overcomes the limitations of traditional U-Net and other networks with their single network structure and loss functions that only focus on pixel precision, achieving the collaborative reconstruction of artistic details and stylistic characteristics of calligraphy and painting images.

[0121] In summary, this invention discloses an image reconstruction device that acquires an ancient painting or calligraphy image to be reconstructed, performs multi-scale degradation simulation on the image, uses the degradation simulation results to perform adaptive degradation correction on the image to be reconstructed to obtain a preprocessed image to be reconstructed, performs blurry detail feature enhancement processing on the preprocessed image to be reconstructed, extracts a target detail enhancement feature map from the enhanced preprocessed image to be reconstructed, and inputs the target detail enhancement feature map into a pre-trained ancient painting or calligraphy image reconstruction model for painting or calligraphy image reconstruction processing to obtain a reconstructed clear ancient painting or calligraphy image. This invention simulates the degradation of original ancient calligraphy and painting images at multiple scales and uses the simulation results to adaptively correct the degradation of the images to be reconstructed. This effectively addresses the non-uniform degradation phenomena in space and scale, enhancing the detail and color information of the images. By enhancing the blurred detail features of the preprocessed images to be reconstructed, it effectively suppresses noise interference while selectively enhancing blurred details, enhancing subtle artistic details while preserving the overall structural information of the image. Furthermore, by introducing a quaternary loss function that integrates pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss during the training of the ancient calligraphy and painting image reconstruction model, it achieves the goal of improving image clarity while maintaining the original brushwork spirit and aesthetic characteristics.

[0122] In one embodiment, the preprocessing unit 302 can specifically be used for: For the ancient calligraphy and painting image to be reconstructed, at each spatial location and preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales; The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image to be reconstructed.

[0123] In one embodiment, the feature enhancement unit 303 can specifically be used for: For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated. Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region; The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map.

[0124] In one embodiment, the image reconstruction apparatus may further include: The training set construction unit is used to construct a training dataset for the reconstruction of ancient calligraphy and painting images. Each sample in the training dataset is an ancient calligraphy and painting image pair, which includes: an original ancient calligraphy and painting image with varying degrees of degradation, and a high-quality, clear reference image corresponding to the original ancient calligraphy and painting image. The degradation correction unit is used to perform multi-scale degradation simulation on the original ancient calligraphy and painting images in each sample of the training dataset, and to perform adaptive degradation correction on the original ancient calligraphy and painting images using the degradation simulation results to obtain a preprocessed image. An enhancement processing unit is used to perform blurred detail feature enhancement processing on the preprocessed image and extract the detail enhancement feature map from the enhanced preprocessed image; The model training unit is used to train the training dataset using a supervised learning method based on the detailed enhancement feature map to obtain the ancient calligraphy and painting image reconstruction model.

[0125] In one embodiment, the degradation correction unit can be specifically used for: For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image.

[0126] In one embodiment, the enhancement processing unit may specifically be used for: For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel; Calculate the local enhancement factor based on the texture complexity and detail richness of the local region; The multi-directional gradient magnitude, the channel fusion weight, and the local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate a detail-enhanced feature map.

[0127] In one embodiment, the model training unit can be specifically used for: The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values; A reconstructed, clear image of calligraphy and painting is generated based on the aforementioned detail enhancement values; Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed. For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is used for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

[0128] In one embodiment, the model training unit can also be used for: A set of weights is calculated for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates. The detail enhancement feature map is subjected to parallel convolution operations using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates; The convolution results of the dilated convolution branches with different dilation rates are multiplied by their corresponding attention weights and summed. The summation result is then added to the bias term of the corresponding output channel to obtain the detail enhancement value.

[0129] In one embodiment, the model training unit can also be used for: The detail enhancement map, composed of the aforementioned detail enhancement values, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstructed image. Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and the reconstructed clear calligraphy and painting image is generated by combining a fully connected network.

[0130] It should be noted that for the specific working principles of each component in the device embodiment, please refer to the corresponding section of the method embodiment, which will not be repeated here.

[0131] Corresponding to the above embodiments, the present invention also discloses a computer storage medium that stores at least one instruction, which, when executed by a processor, implements the steps shown in the embodiments of the image reconstruction method.

[0132] Corresponding to the above embodiments, such as Figure 9 As shown, the present invention also provides a schematic diagram of the structure of an electronic device, which may include: a processor 1 and a memory 2; The processor 1 and memory 2 communicate with each other via communication bus 3. Processor 1, for executing at least one instruction; Memory 2 is used to store at least one instruction; Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0133] Memory 2 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0134] The processor executes at least one instruction to implement the steps shown in the embodiment of the image reconstruction method.

[0135] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0137] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image reconstruction method, characterized in that, include: Obtain images of ancient paintings and calligraphy to be reconstructed; Multi-scale degradation simulation is performed on the ancient calligraphy and painting image to be reconstructed, and adaptive degradation correction is performed on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results to obtain a preprocessed image to be reconstructed. The preprocessed image to be reconstructed is subjected to blurred detail feature enhancement processing, and the target detail enhancement feature map is extracted from the enhanced preprocessed image to be reconstructed. The target detail enhancement feature map is input into a pre-trained ancient calligraphy and painting image reconstruction model for calligraphy and painting image reconstruction processing to obtain a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is obtained by continuously optimizing the model parameters until the stopping iteration condition is met, with the minimization of the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

2. The image reconstruction method according to claim 1, characterized in that, Multi-scale degradation simulation is performed on the ancient calligraphy and painting image to be reconstructed, and adaptive degradation correction is performed on the image using the degradation simulation results to obtain a preprocessed image to be reconstructed, including: For the ancient calligraphy and painting image to be reconstructed, at each spatial location and preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The local degradation kernel is used to simulate the degradation of the ancient calligraphy and painting image to be reconstructed at different scales; The simulation results at various scales are adaptively fused based on the richness of detail in the local areas at each location, while a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image to be reconstructed.

3. The image reconstruction method according to claim 1 or 2, characterized in that, The preprocessed image to be reconstructed is subjected to blurred detail feature enhancement processing, and the target detail enhancement feature map is extracted from the enhanced preprocessed image to be reconstructed, including: For each color channel of the preprocessed image to be reconstructed, calculate the target multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image to be reconstructed and the global mean of the color channel to which the current pixel value belongs, the target channel fusion weight corresponding to each color channel is calculated. Calculate the target local enhancement factor based on the texture complexity and detail richness of the local region; The target multi-directional gradient magnitude, the target channel fusion weight, and the target local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate the target detail enhancement feature map.

4. The image reconstruction method according to claim 1, characterized in that, The training process of the ancient calligraphy and painting image reconstruction model includes: A training dataset for reconstructing ancient calligraphy and painting images is constructed, wherein each sample in the training dataset is an ancient calligraphy and painting image pair, and the ancient calligraphy and painting image pair includes: an original ancient calligraphy and painting image with different degrees of degradation, and a high-quality clear reference image corresponding to the original ancient calligraphy and painting image; Multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and the degradation simulation results are used to adaptively correct the degradation of the original ancient calligraphy and painting images to obtain preprocessed images; The preprocessed image is subjected to blurry detail feature enhancement processing, and the detail enhancement feature map is extracted from the enhanced preprocessed image; Based on the enhanced detail feature maps, the training dataset is trained using a supervised learning method to obtain the ancient calligraphy and painting image reconstruction model.

5. The image reconstruction method according to claim 4, characterized in that, Multi-scale degradation simulation is performed on the original ancient calligraphy and painting images in each sample of the training dataset, and adaptive degradation correction is performed on the original ancient calligraphy and painting images using the degradation simulation results to obtain preprocessed images, including: For each of the original ancient paintings and calligraphy images, at each spatial location and at a preset scale, the convolution kernel representing the local blurring or degradation effect of the spatial location is estimated and used as the local degradation kernel; The original ancient calligraphy and painting image is simulated for degradation at different scales using the local degradation kernel. The simulation results at each scale are adaptively fused according to the richness of detail in the local area at each location. At the same time, a bias term is added to adjust the color for adaptive degradation correction, resulting in the preprocessed image.

6. The image reconstruction method according to claim 4 or 5, characterized in that, The preprocessed image is subjected to blurry detail feature enhancement processing, and the detail enhancement feature map is extracted from the enhanced preprocessed image, including: For each color channel of the preprocessed image, calculate the multi-directional gradient magnitude; Based on the absolute deviation between the current pixel value of the preprocessed image and the global mean of the color channel to which the current pixel value belongs, calculate the channel fusion weight corresponding to each color channel; Calculate the local enhancement factor based on the texture complexity and detail richness of the local region; The multi-directional gradient magnitude, the channel fusion weight, and the local enhancement factor are combined, and a non-linear activation operation is performed with the addition of a bias term to generate a detail-enhanced feature map.

7. The image reconstruction method according to claim 4, characterized in that, Based on the aforementioned detail-enhanced feature maps, a supervised learning method is used to train the training dataset to obtain the ancient calligraphy and painting image reconstruction model, including: The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values; A reconstructed, clear image of calligraphy and painting is generated based on the aforementioned detail enhancement values; Based on the reconstructed clear calligraphy and painting image and the corresponding high-quality clear reference image, a quaternary loss function is constructed. For the training dataset of ancient calligraphy and painting image reconstruction, a supervised learning method is used for training. The training objective is to minimize the quaternary loss function. The model parameters are continuously optimized until the stopping iteration condition is met, and the ancient calligraphy and painting image reconstruction model is obtained.

8. The image reconstruction method according to claim 7, characterized in that, The detail enhancement feature map is processed using a detail enhancement convolution module to obtain detail enhancement values, including: A set of weights is calculated for each spatial location and each output channel of the detail enhancement feature map to obtain the attention weights corresponding to the dilated convolution branches with different dilation rates. The detail enhancement feature map is subjected to parallel convolution operations using multiple dilated convolution kernels with different dilation rates to obtain convolution results of dilated convolution branches with different dilation rates; The convolution results of the dilated convolution branches with different dilation rates are multiplied by their corresponding attention weights and summed. The summation result is then added to the bias term of the corresponding output channel to obtain the detail enhancement value.

9. The image reconstruction method according to claim 7, characterized in that, Based on the aforementioned detail enhancement values, a reconstructed clear image of the calligraphy or painting is generated, including: The detail enhancement map, composed of the aforementioned detail enhancement values, is concatenated with the preprocessed image along the channel dimension and then input into the initial reconstruction sub-network mapping function to obtain the initial reconstructed image. Based on the initial reconstructed image, an encoding operation is performed using an autoencoder encoding model and a decoding operation is performed using a decoding model, and the reconstructed clear calligraphy and painting image is generated by combining a fully connected network.

10. An image reconstruction apparatus, characterized in that, include: The image acquisition unit is used to acquire images of ancient paintings and calligraphy to be reconstructed. The preprocessing unit is used to perform multi-scale degradation simulation on the ancient calligraphy and painting image to be reconstructed, and to perform adaptive degradation correction on the ancient calligraphy and painting image to be reconstructed using the degradation simulation results, so as to obtain the preprocessed image to be reconstructed. The feature enhancement unit is used to perform blurred detail feature enhancement processing on the preprocessed image to be reconstructed, and extract the target detail enhancement feature map from the enhanced preprocessed image to be reconstructed; The image reconstruction unit is used to input the target detail enhancement feature map into a pre-trained ancient calligraphy and painting image reconstruction model to perform calligraphy and painting image reconstruction processing, thereby obtaining a reconstructed clear ancient calligraphy and painting image. The ancient calligraphy and painting image reconstruction model is trained by continuously optimizing the model parameters until the stopping iteration condition is met, with minimizing the quaternary loss function as the training objective. The quaternary loss function is a loss function that combines pixel fidelity loss, multi-scale perception loss, local brushstroke coherence loss, and spirit consistency loss.

11. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which, when executed by a processor, implements the image reconstruction method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, The electronic device includes: a memory and a processor; The memory is used to store at least one instruction; The processor is used to execute the at least one instruction to implement the image reconstruction method as described in any one of claims 1 to 9.

Images