Evaluation method and system applied to restoration image of ancient Chinese calligraphy and painting

Abstract

The invention relates to an evaluation method and system applied to restored images of ancient Chinese calligraphy and painting, and the method comprises the steps: 1, taking the restored images of a first ancient calligraphy and painting and a second ancient calligraphy and painting, which are restored through an artificial intelligence technology and PS software, as evaluation preprocessing samples, selecting a peak signal-to-noise ratio, a structural similarity index and a color difference to combine with a subjective visual MOS score to evaluate a restoration result; 2, according to the evaluation condition, optimizing the weight of each feature component in the ancient calligraphy and painting repair evaluation for one objective evaluation index in the step 1, and the like. The evaluation system comprises an ancient calligraphy and painting repairing module, an evaluation module and a verification module. The invention further provides electronic equipment and a computer readable storage medium which are applied to the evaluation method and system. The method has the advantages that the restoration result of the ancient calligraphy and painting is assessed through a multi-dimensional assessment system, four indexes of PSNR, SSIM, HSV and MOS are fused, and the texture, color, structure and human eye perception full-dimensional requirements of restoration of the ancient calligraphy and painting are covered.

Description

Technical Field

[0001] This invention belongs to the field of ancient calligraphy and painting restoration technology, specifically relating to an evaluation method and system for the restoration of images of ancient Chinese calligraphy and paintings. Background Technology

[0002] Ancient calligraphy and paintings, as an important material cultural heritage of Chinese civilization, possess irreplaceable value in the preservation and transmission of historical culture through their restoration. Traditional evaluation of the restoration effects of ancient calligraphy and paintings relies primarily on the subjective experience of restoration experts, lacking an objective and unified evaluation system. This leads to problems such as strong subjectivity and poor reproducibility in restoration quality assessment. In recent years, although digital image processing technology has been applied to the field of cultural relic restoration, and some Chinese scholars have conducted in-depth research based on artificial intelligence technology, resulting in AI restoration algorithms such as ACP-LaMa and progressive restoration based on multi-level features (suitable for large datasets), or direct restoration using applications such as Photoshop (suitable for single images and highly efficient), significant technological gaps remain in the evaluation of restoration effects for ancient calligraphy and paintings, a special type of cultural relic.

[0003] In existing technologies, whether it is the aforementioned artificial intelligence algorithms or software restoration methods, the evaluation of conventional image restoration effects often adopts a single index analysis, such as focusing only on the peak signal-to-noise ratio represented by the PSNR value or the structural similarity reflected by the SSIM value, which makes it difficult to fully reflect the special requirements of ancient calligraphy and painting restoration, such as the coordination of background color and the continuity of brushstrokes.

[0004] For example, Chinese invention patent application publication number CN116542953A discloses an invention entitled "A Method and System for Digital Image Restoration of Ancient Paintings." This method involves acquiring a digital image of the ancient painting to be restored, analyzing the image to determine its damage and color information, and formulating a restoration plan based on this information. The restoration plan includes an algorithmic restoration method, algorithmic restoration standards, and human visual judgment requirements. The method then performs damage restoration processing on the digital image. After restoration, the results are evaluated using the algorithmic restoration standards and human visual judgment requirements. If the evaluation results indicate that the restoration meets both the algorithmic restoration standards and human visual judgment requirements, the digital image restoration is deemed successful. However, this invention does not provide specific algorithmic restoration standards, nor does it optimize the evaluation criteria for the aging of ancient painting materials and the fading of pigments. This results in insufficient sensitivity of the algorithmic restoration standards to the texture of Xuan paper and the characteristics of ink wash. More importantly, these studies or evaluation methods often only use universal image quality indicators or subjective visual evaluations for verification, without taking into account the consistency issues in the subjective and objective evaluation of restored images.

[0005] The reliability of existing objective evaluation indicators for the restoration assessment of ancient calligraphy and paintings needs further analysis and optimization. Some scholars in China have conducted research on the restoration of ancient calligraphy and paintings using artificial intelligence technology, resulting in AI restoration algorithms such as ACP-LaMa and progressive algorithms based on multi-level feature restoration. These AI-based restoration algorithms are mostly suitable for the restoration of ancient calligraphy and paintings with relatively large datasets. Other scholars directly use applications such as Photoshop for digital restoration of ancient calligraphy and paintings; this method is suitable for single-image restoration and is highly efficient. Regardless of the restoration method, scholars have ultimately verified the effectiveness of their methods either using universal image quality evaluation indicators or directly verifying the restoration effect through subjective visual evaluation. However, these studies have not considered the consistency between subjective and objective evaluation of restored images. The reliability of existing objective evaluation indicators for the restoration assessment of ancient calligraphy and paintings needs further analysis and optimization to develop a dedicated art evaluation system for the restoration assessment of ancient calligraphy.

[0006] Therefore, an effective correlation between objective evaluation indicators and subjective visual perception has not yet been established, which means that each restoration needs to rely on manual judgment, increasing the workload of experts and scholars, and there is also a lack of an art evaluation system specifically for the restoration and assessment of ancient calligraphy and paintings.

[0007] In view of the above-mentioned technical problems existing in the prior art, the present invention provides an evaluation method and system for the restoration of ancient Chinese calligraphy and paintings. Summary of the Invention

[0008] This invention provides an evaluation method and system for the restoration of images of ancient Chinese calligraphy and paintings.

[0009] The evaluation method applied to the restoration of ancient Chinese paintings and calligraphy includes:

[0010] Step 1: The restored images of the first and second ancient calligraphy and paintings, restored using artificial intelligence technology and Photoshop software, are used as preprocessing samples for evaluation. At the same time, three existing objective evaluation indicators are selected: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Hue, Saturation, Value (HSV), which are combined with subjective visual MOS scores to evaluate the restoration results.

[0011] Step 2: Based on the assessment and restoration results, optimize the weights of each feature component of the one objective assessment indicator from Step 1, namely the Structural Similarity Index (SSIM), in the assessment of ancient calligraphy and painting restoration.

[0012] Step 3: Use Photoshop and AI generation tools to formally restore the missing parts and background color of the ancient calligraphy and painting image. Use four objective evaluation methods before and after optimization and subjective MOS scores to evaluate the restoration results, verify the effectiveness of the restoration, and further verify the correlation and consistency between the optimization indicators and the subjective results.

[0013] Furthermore, in step 1, this preprocessing uses two examples: an M-grid ancient character restored based on artificial intelligence technology and an ancient character example provided by the museum restored using PS software. The images before and after restoration of both examples need to be uniformly cropped into M regions, which are labeled as 0 to M-1 as numbers.

[0014] Further, in step 1, three objective evaluation indicators, PSNR, SSIM, and HSV, are selected. Combined with the user's subjective visual evaluation MOS score of the restored image, each region of the two examples is evaluated, resulting in three sets of objective evaluation values ​​and one set of subjective evaluation MOS scores for each region of the two examples before and after restoration.

[0015] Furthermore, for the three objective evaluation indicators PSNR, SSIM, and HSV, and a set of subjective evaluation MOS scores, PSNR is used to evaluate the degree of distortion of the images before and after restoration. The higher the value, the better the restoration effect.

[0016] SSIM comprises three components: luminance similarity (l), contrast similarity (c), and structural similarity (s), with values ​​ranging from 0 to 1. The closer the values ​​of the images before and after restoration are to 1, the better the restoration effect. HSV is used to evaluate the difference in background color between the images before and after restoration. After processing such as computer code unit conversion, its value ranges from 0 to 1. The closer it is to 0, the smaller the color difference between the two images, and the better the restoration effect. The restoration results are subjectively scored, with scores ranging from 1 to 5, representing very dissatisfied, dissatisfied, neutral, satisfied, and very satisfied, respectively. The MOS value is the average of all user scores.

[0017] Furthermore, in step 2, the weights of each feature component of SSIM are optimized based on the ancient calligraphy and painting restoration assessment results in step 1. This includes: analyzing the impact of each feature component of SSIM on the assessment in different regions for two different examples and optimizing the SSIM index; and verifying the correlation and consistency between PSNR and subjective MOS value, SSIM and subjective MOS value before optimization, SSIM and subjective MOS value after optimization, and HSV color difference and subjective MOS value.

[0018] Furthermore, in step 2, the influence of each feature component of SSIM on the evaluation is analyzed in different regions for the two types of samples, and the SSIM index is optimized. The specific steps are as follows:

[0019] Step 2.1.1 Extract the SSIM component values ​​l (luminance similarity), c (contrast similarity), and s (structure similarity) for each region of the two types of samples before and after restoration. Compare the changing trends of l with SSIM, c with SSIM, and s with SSIM. For ancient calligraphy and painting images restored using artificial intelligence technology or Photoshop software, the influence of the three components in the SSIM evaluation is c>l>s, specifically:

[0020] In the evaluation of the restored ancient characters in the M-grid image, if M=6, the SSIM values ​​of each region before and after restoration are ranked as follows: 0.6534 (number 3) > 0.6402 (number 0) > 0.6317 (number 4) > 0.6250 (number 2) > 0.6147 (number 1) > 0.552 (number 5). The l-component of the SSIM values ​​of each region before and after restoration is ranked as follows: 0.9673 (number 0) > 0.966 (number 1) > 0.9652 (number 2) > 0.9648 (number 4) > 0.9612 (number 3) > 0.9569 (number 5). The order of the c component of the SSIM values ​​before and after region restoration is 0.3851 (number 3) > 0.3777 (number 0) > 0.3704 (number 4) > 0.352 (number 2) > 0.3377 (number 1) > 0.2876 (number 5). The order of the s component of the SSIM values ​​is 10.114 (number 1) > 10.0455 (number 3) > 9.9878 (number 2) > 9.8555 (number 5) > 9.7766 (number 0) > 8.99 (number 4). At this point, c and SSIM show the same trend, then l is closer, and finally s.

[0021] Step 2.1.2 Set the sum of the three components to 1. According to the weight tendency of c>l>s in Step 2.1.1, further allocate the weights. Based on the existence of the three components l, s, and c in SSIM in Step 2.1.1, the weight combinations are: 3:5:2, 4:5:1, 3:6:1, 4:6:0, 2:7:1, 3:7:0, 2:8:0, 1:9:0;

[0022] Step 2.1.3 Using the gray-level co-occurrence matrix, extract the texture maps of the three features l, c, and s of each region after the two types of sample repairs according to the weights in Step 2.2.2;

[0023] Step 2.1.4 Use SSIM and PSNR to evaluate the texture maps and repaired images of each region in the two examples (since the gray-level co-occurrence matrix extracts gray-level images and does not involve color, HSV color difference evaluation is not required).

[0024] Step 2.1.5 Identify the feature combination corresponding to the optimal evaluation result for each region in the two types of samples in 2.1.4, and calculate the normalized results of the three feature components of SSIM according to the following formula (1):

[0025]

[0026] In equation (1): x is the l, s, c weight value corresponding to the optimal evaluation result of the Mth region of the two types of samples using SSIM and PSNR in step 2.4; S is the standard deviation of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.1.4; μ is the average value of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.1.4. M The value of x after normalization is the final weight of each feature in the Mth region after the two samples are calculated separately;

[0027] Step 2.1.6 Substitute the weight values ​​calculated in Step 2.1.5 into the following equation (2) to calculate the optimized SSIM value:

[0028]

[0029] In equation (2), SSIM(X,Y) represents the SSIM value of the entire image before and after restoration, l(X,Y) represents their brightness similarity, c(X,Y) represents their contrast similarity, s(X,Y) represents their contrast similarity, M represents the image segmentation block, for example, when the sample is a 6-grid ancient character, M=6, M is determined according to the initial number of segmentation blocks; α represents the brightness similarity weight, since brightness determines the whole, it is not calculated according to the block; β represents the contrast similarity weight; γ represents the structural similarity weight. At this time, when x in equation (1) M When the result is the result after normalization of l: α M =x M When x in equation (1) M When the result is normalized to c, it must satisfy: β1 + β2 + ... + βj = x1 + x2 + ... + x M =1, when x in equation (1) M When the result is the normalized result of s, it must satisfy γ1+γ2+···+γj=x1+x2+···+x M =1.

[0030] Further, in step 2, the correlation and consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value are verified respectively. This includes: using the Pearson Correlation Coefficient (PLCC) to verify the correlation between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value respectively; and using the Root Mean Square Error (RMSE) to verify the consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value respectively.

[0031] Furthermore, in step 2, the Pearson Correlation Coefficient (PLCC) is used to verify the correlation between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value, respectively. The steps are as follows:

[0032] Step 2.2.1 Substitute the weights of each component obtained in step 2.1.5 into the above formula (2) to calculate the SSIM (optimized) values ​​before and after the whole frame restoration of the two samples respectively;

[0033] Step 2.2.2 Calculate the PSNR and its subjective MOS value, SSIM (before optimization) and subjective MOS value, and HSV color difference and subjective MOS value of the two sample images before and after restoration, obtained in Step 1, according to the following formula (3), 3 sets; and the SSIM (after optimization) and subjective MOS value of the two sample images before and after restoration, obtained in Step 2.2.1, 1 set:

[0034]

[0035] In equation (3), N represents the sample size, s i This represents the MOS value, which represents the rating of each sample image by all users. p represents the average MOS score of all users on N sample images; i This represents the evaluation value of a certain objective indicator for each sample image. This represents the average value of an objective indicator for N sample images; calculate the PLCC values ​​of the four objective indicators and the subjective MOS value for the images before and after restoration in two different examples, and compare the correlation between the subjective and objective indicators.

[0036] Step 2.2.3 The range of the four PLCC values ​​is between [-1, 1]. The closer the absolute value of PLCC is to 1, the higher the correlation between the objective indicator and the subjective MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the PLCC values ​​of the four objective indicators and the subjective MOS value are all 1, and the RMSE value needs to be compared further.

[0037] Furthermore, in step 2, the consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value are verified using the root mean square error (RMSE). The steps are as follows:

[0038] Step 2.3.1 Calculate the RMSE values ​​of the PSNR and its subjective MOS value, SSIM (before optimization) and subjective MOS value, and HSV color difference and subjective MOS value of the two sample images before and after restoration obtained in Step 1 according to the following formula (4), 3 sets; and the RMSE values ​​of SSIM (after optimization) and subjective MOS value of the two sample images before and after restoration obtained in Step 2.2.1, 1 set:

[0039]

[0040] In equation (4), N represents the sample size, s i p represents the MOS value, which represents the rating of each sample image by all users; i This represents the evaluation value of a certain objective indicator for each sample image; the RMSE values ​​of the four objective indicators and the subjective MOS value are calculated for the images before and after restoration of the two sample images respectively, and the consistency between the subjective and objective indicators is compared.

[0041] Step 2.3.2 The four sets of RMSE values ​​range from [0, z), where z is related to the range of different subjective and objective evaluation indicators. The closer the RMSE is to 0, the higher the consistency between the objective indicator and the subjective MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the RMSE value of SSIM (after optimization) is 0.3202, and the RMSE value of SSIM (before optimization) is 0.3448. The RMSE value after optimization is 0.0246 lower than that before optimization, indicating that the optimized SSIM is effective. Based on SSIM, an objective evaluation indicator that is more reliable than the other three objective indicators in the evaluation of ancient character restoration is generated. The evaluation preprocessing is completed, and the formal evaluation stage, step 3, begins.

[0042] Furthermore, in step 3, the missing parts and background color of the ancient calligraphy and painting images are repaired using Photoshop and AI generation tools, including: using Photoshop to supplement the missing edges and seal parts of the ancient characters provided by the museum; and using AI generation tools to supplement the missing background color of the ancient characters provided by the museum.

[0043] Furthermore, in step 3, the missing edges of the ancient characters provided by the museum are supplemented using Photoshop, including the following steps:

[0044] Step 3.1 Complete the text: Place the missing sample and rubbing provided by the museum into the Photoshop canvas, duplicate the rubbing layer, adjust the rubbing's opacity and place it above the missing sample; use the free transform tool to transform the rubbing, and fill in the missing parts of the sample by changing the size of the rubbing; after each part is completed, use the lasso tool or selection tool to select the completed part of the rubbing (including the background), and create new layers for the selected parts; name the new layers, enlarge each missing part and adjust its position and details to ensure a complete connection with the missing part of the sample; merge the text layers and the chapter layers;

[0045] Step 3.2 Complete the stamp: Using the "Channels" function in Photoshop, adjust the image's levels, contrast, saturation, etc., to ensure the stamp's color matches the missing original image. Figure 1 Therefore, after steps 3.1 and 3.2, we obtain the image of the ancient character with the missing content filled in.

[0046] Furthermore, in step 3, the missing background color of the ancient characters provided by the museum is supplemented using an AI generation tool. This includes: importing the image of the ancient characters with the supplemented content into the AI ​​generation tool, selecting the expansion tool, framing the part where the background color needs to be supplemented, and similarly using the brush to frame other areas of the image where the background color can be selected. This allows the background color content of other parts to be automatically filled into the part that needs to be supplemented.

[0047] Furthermore, in step 3, four objective evaluation indicators, namely PSNR, SSIM (before optimization), SSIM (after optimization), and HSV color difference, are used to evaluate the formal restoration effect in combination with the user's subjective visual evaluation MOS score of the restored image. Four sets of objective evaluation values ​​and one set of subjective evaluation MOS scores are obtained for the missing original image and the rubbing, the rubbing and the restored image, and the missing original image and the restored image.

[0048] The final restoration effect was evaluated by combining the user's subjective visual assessment MOS score of the restored image. The restored image was compared with the missing original image. Except for the HSV color difference, the PSNR, SSIM value before and after optimization, and subjective MOS value were all significantly improved compared with the original image, indicating that the restoration was completely effective. Specifically: the PSNR of the restored image was 6.0646 higher than that of the original image, indicating that the restored image was clearer and less distorted than the rubbing; the SSIM (before optimization) was 0.1569 higher than that of the original image, and the SSIM (after optimization) was 0.1838 higher than that of the original image, indicating that the structural similarity between the restored image and the missing original image was better than that of the rubbing; the HSV color difference was 0.0342 lower than that of the original rubbing, indicating that the background color of the restored image was closer to that of the original image than the rubbing.

[0049] Further, in step 3, the PSNR and subjective MOS values ​​of the missing original image and rubbing, the rubbing and restored image, the missing original image and restored image, the SSIM (before optimization) and subjective MOS values, the SSIM (after optimization) and subjective MOS values, and the PLCC and RSME values ​​of HSV color difference and subjective MOS values ​​are calculated to verify the correlation and consistency between the four objective evaluation indicators and subjective evaluation in ancient character restoration:

[0050] The results show that the absolute PLCC value of SSIM (after optimization) and MOS is the largest, exceeding the absolute PLCC value of SSIM (before optimization) and MOS by 0.1363, exceeding the absolute PLCC value of PSNR and MOS by 0.2157, and exceeding the absolute PLCC value of HSV color difference and MOS by 0.4482, indicating that the correlation between SSIM (after optimization) and MOS is the best. At the same time, the RMSE value of SSIM (after optimization) and MOS is the smallest, 0.0095 smaller than that before optimization, indicating the best consistency between the two. Furthermore, SSIM (after optimization) is the indicator that is closest to the subjective evaluation among the three objective indicators. This also verifies that in the results of the formal restoration evaluation, the SSIM (after optimization) value of the restored image and the missing original image is closer to 1 than that of the restored image and the rubbing, and the rubbing and the missing original image, indicating that the restoration is effective.

[0051] Secondly, the present invention also provides an evaluation system for the restoration of images of ancient Chinese paintings and calligraphy, comprising:

[0052] Ancient calligraphy and painting restoration module: Uses artificial intelligence technology, AI tools or Photoshop software to restore missing ancient calligraphy and paintings;

[0053] Evaluation module: The repair results are evaluated using five evaluation methods: PSNR, SSIM (before optimization), SSIM (after optimization), HSV color difference, and subjective MOS score.

[0054] Verification / Evaluation Comparison Module: This module compares and verifies the results of the objective evaluation methods described above with the user's subjective visual evaluation (i.e., the results of the five evaluation methods in the evaluation module), providing a reference for the restoration evaluation of ancient calligraphy and paintings by museums or related art institutions.

[0055] Thirdly, the present invention also provides an electronic device for evaluating the restoration image of the ancient Chinese calligraphy and painting, the electronic device including a memory and a processor, the memory storing a computer program capable of running on the processor, and the processor executing the computer program to implement the various steps of the evaluation method for the restoration image of the ancient Chinese calligraphy and painting.

[0056] Fourthly, the present invention further provides an evaluation method and system for the restoration of ancient Chinese paintings and calligraphy, and a computer-readable storage medium for the electronic device, wherein the computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the method described in any of the first aspects above.

[0057] Compared with the prior art, the superior effects of the present invention are as follows:

[0058] 1. The evaluation method for the restoration of ancient Chinese calligraphy and painting images described in this invention evaluates the restoration results of ancient calligraphy and painting through a multi-dimensional evaluation system, integrating four core indicators: PSNR, SSIM, HSV and MOS, covering all dimensions of the restoration of ancient calligraphy and painting, including texture, color, structure and human visual perception.

[0059] 2. The evaluation method for the restoration of ancient Chinese paintings and calligraphy described in this invention improves the consistency between the SSIM algorithm's judgment on the restoration effect of ancient paintings and calligraphy and MOS by more than 40% based on objective evaluation indicators and MOS optimization, which is significantly better than the unoptimized SSIM.

[0060] 3. The evaluation method for the restoration of ancient Chinese calligraphy and paintings described in this invention uses PLCC and RMSE values ​​to verify the correlation and consistency between several objective indicators and subjective evaluation. It proves that the optimized SSIM value has the best correlation and consistency with subjective evaluation when evaluating the restoration of ancient calligraphy and paintings, and is the most reliable.

[0061] 4. The evaluation method and system for the restoration of ancient Chinese calligraphy and paintings described in this invention applies academic research on the evaluation of the restoration of ancient calligraphy and paintings to practice, forming a system specifically for evaluating the restoration effect of ancient calligraphy and paintings, and satisfying the objective judgment of the restoration effect of ancient calligraphy and paintings by museums or related art institutions. Attached Figure Description

[0062] Figure 1This is a flowchart illustrating the evaluation method for the restoration of ancient Chinese calligraphy and paintings as described in this invention.

[0063] Figure 2 This is a schematic diagram of the structure of the evaluation system for the restoration of ancient Chinese paintings and calligraphy, as described in this invention.

[0064] Figure 3 This is a schematic diagram of the architecture of the electronic device used in the evaluation method and system for restoring images of ancient Chinese paintings and calligraphy. Detailed Implementation

[0065] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0066] Example 1

[0067] like Figure 1 As shown, the evaluation method for restoring images of ancient Chinese calligraphy and paintings includes the following steps:

[0068] Step 1: The restored images of the first and second ancient calligraphy and paintings, restored using artificial intelligence technology and Photoshop software, are used as preprocessing samples for evaluation. At the same time, three existing objective evaluation indicators are selected: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Hue, Saturation, Value (HSV), which are combined with subjective visual MOS scores to evaluate the restoration results.

[0069] Step 2: Based on the assessment and restoration results, optimize the weights of each feature component in the ancient calligraphy and painting restoration assessment using one of the objective assessment indicators SSIM from Step 1.

[0070] Step 3: Use Photoshop and AI generation tools to formally restore the missing parts and background color of the ancient calligraphy and painting image. Use four objective evaluation methods before and after optimization and subjective visual MOS score values ​​to evaluate the restoration results, verify the effectiveness of the restoration, and further verify the correlation and consistency between the optimization indicators and the subjective results.

[0071] As a specific step of the present invention, in step 1, two examples are used in this preprocessing: M-grid ancient characters restored based on artificial intelligence technology and ancient characters provided by the museum restored by PS software; and the images before and after restoration of the two examples are uniformly cropped into M regions, which are marked as 0 to M-1 as numbers.

[0072] As a specific step of the present invention, in step 1, three objective evaluation indicators, PSNR, SSIM and HSV, are selected. Combined with the user's MOS score of the restored image, each region of the two examples is evaluated, and three sets of objective evaluation values ​​and one set of MOS scores are obtained for each region of the two examples before and after restoration.

[0073] As a specific step of this invention, in step 1, three objective evaluation indicators—PSNR, SSIM, and HSV—and a set of MOS scores are used. PSNR is used to evaluate the degree of distortion in the images before and after restoration; the higher the value, the better the restoration effect. SSIM includes three components: luminance similarity (l), contrast similarity (c), and structural similarity (s), with values ​​between 0 and 1. The closer the values ​​of the images before and after restoration are to 1, the better the restoration effect. HSV color difference is used to evaluate the difference in background color between the images before and after restoration. After computer code unit conversion and other processing, its value ranges from 0 to 1; the closer it is to 0, the smaller the color difference between the two images, and the better the restoration effect. The restoration results are subjectively scored, with scores ranging from 1 to 5, representing very dissatisfied, dissatisfied, neutral, satisfied, and very satisfied, respectively. The MOS value is the average of all user scores.

[0074] As a specific step of the present invention, in step 2, the weights of each feature component of SSIM are optimized based on the ancient calligraphy and painting restoration assessment in step 1, including: analyzing the influence of each feature component of SSIM on the assessment in different regions of the two examples and optimizing the SSIM index; verifying the correlation and consistency between PSNR and subjective MOS value, SSIM and subjective MOS value before optimization, SSIM and subjective MOS value after optimization, and HSV color difference and subjective MOS value.

[0075] As a specific step of the present invention, in step 2, the influence of each feature component of SSIM on the evaluation is analyzed in two examples under different regions, and the SSIM index is optimized. The specific steps are as follows:

[0076] Step 2.1.1 Extract the SSIM component values ​​l (luminance similarity), c (contrast similarity), and s (structure similarity) for each region of the two types of samples before and after restoration. Compare the changing trends of l with SSIM, c with SSIM, and s with SSIM. For ancient calligraphy and painting images restored using artificial intelligence technology or Photoshop software, the influence of the three components in the SSIM evaluation is c>l>s, specifically:

[0077] In the evaluation of the restored ancient characters in the M-grid image, if M=6, the SSIM values ​​of each region before and after restoration are ranked as follows: 0.6534 (number 3) > 0.6402 (number 0) > 0.6317 (number 4) > 0.6250 (number 2).

[0078] The order of the l-component values ​​of the SSIM values ​​before and after restoration for each region is: 0.9673 (No. 0) > 0.966 (No. 1) > 0.9652 (No. 2) > 0.9648 (No. 4) > 0.9612 (No. 3) > 0.9569 (No. 5). The order of the c-component values ​​of the SSIM values ​​before and after restoration for each region is: 0.3851 (No. 3) > 0.3777 (No. 0). The SSIM values ​​are ranked as follows: 0.114 (number 4) > 0.3704 (number 4) > 0.352 (number 2) > 0.3377 (number 1) > 0.2876 (number 5). The SSIM values ​​are ordered as follows: 10.114 (number 1) > 10.0455 (number 3) > 9.9878 (number 2) > 9.8555 (number 5) > 9.7766 (number 0) > 8.99 (number 4). At this point, c follows the same trend as SSIM, then l is closer, and finally s.

[0079] Step 2.1.2 Set the sum of the three components to 1. According to the weight tendency of c>l>s in Step 2.1.1, further allocate the weights. Based on the existence of the three components l, s, and c in SSIM in Step 2.1.1, the weight combinations are: 3:5:2, 4:5:1, 3:6:1, 4:6:0, 2:7:1, 3:7:0, 2:8:0, 1:9:0;

[0080] Step 2.1.3 Using the gray-level co-occurrence matrix, extract the texture maps of the three features l, c, and s of each region after the two types of sample repairs according to the weights in Step 2.2.2;

[0081] Step 2.1.4 Use SSIM and PSNR to evaluate the texture maps and repaired images of each region in the two examples (since the gray-level co-occurrence matrix extracts gray-level images and does not involve color, HSV color difference evaluation is not required at this time).

[0082] Step 2.1.5 Identify the feature combination corresponding to the optimal evaluation result for each region in the two types of samples in 2.1.4, and calculate the normalized results of the three feature components of SSIM according to the following formula (1):

[0083]

[0084] In equation (1): x is the l, s, c weight value corresponding to the optimal evaluation result of the Mth region of the two types of samples using SSIM and PSNR in step 2.1.4, S is the standard deviation of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.4, and μ is the average value of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.1.4. M The value of x after normalization is the final weight of each feature in the Mth region after the two samples are calculated separately;

[0085] Step 2.1.6 Substitute the weights calculated in Step 2.1.5 into the following equation (2) to calculate the optimized SSIM value:

[0086]

[0087] In equation (2), SSIM(X,Y) represents the SSIM value of the entire image before and after restoration, l(X,Y) represents their brightness similarity, c(X,Y) represents their contrast similarity, s(X,Y) represents their contrast similarity, M represents the image segmentation block, for example, when the sample is a 6-grid ancient character, M=6, M is determined according to the initial number of segmentation blocks; α represents the brightness similarity weight, since brightness determines the whole, it is not calculated according to the block; β represents the contrast similarity weight; γ represents the structural similarity weight. At this time, when x in equation (1) M When the result is the result after normalization of l: α M =x M When x in equation (1) M When the result is the result of normalization of c, the following must be satisfied:

[0088] β1+β2+···+βj=x1+x2+···+xM=1, when x in equation (1) M When the result is the result of normalization of s, it must satisfy the following condition.

[0089] γ1+γ2+···+γj=x1+x2+···+x M =1.

[0090] As a specific step of the present invention, step 2 verifies the correlation and consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value, respectively. This includes: using the Pearson Correlation Coefficient (PLCC) to verify the correlation between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value, respectively; and using the Root Mean Square Error (RMSE) to verify the consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value, respectively.

[0091] As a specific step of this invention, in step 2, the correlation between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value are verified using the Pearson correlation coefficient, respectively. The steps are as follows:

[0092] Step 2.2.1 Substitute the weights of each component obtained in step 2.1.5 into the above formula (2) to calculate the SSIM (optimized) values ​​before and after the whole frame restoration of the two samples respectively;

[0093] Step 2.2.2 Calculate the PSNR and its subjective MOS value, SSIM (before optimization) and subjective MOS value, HSV color difference and PLCC value of the two sample images before and after restoration obtained in Step 1 according to the following formula (3), 3 sets; and the SSIM (after optimization) and PLCC value of the two sample images before and after restoration obtained in Step 2.2.1, 1 set:

[0094]

[0095] In equation (3), N represents the sample size, s i This represents the MOS value, which represents the rating of each sample image by all users. p represents the average MOS score of all users on N sample images; i This represents the evaluation value of a certain objective indicator for each sample image. This represents the average value of an objective indicator for N sample images; calculate the PLCC values ​​of the three objective indicators PSNR, SSIM, HSV and MOS for the images before and after restoration in two different examples, and compare the correlation between subjective and objective indicators.

[0096] Step 2.2.3 The four sets of PLCC values ​​are in the range of [-1, 1]. The closer the absolute value of PLCC is to 1, the higher the correlation between the objective indicator and the subjective MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the PLCC values ​​of the four objective indicators and the subjective MOS value are all 1, and the RMSE value needs to be compared further.

[0097] As a specific step of this invention, in step 2, the consistency between PSNR and subjective MOS value, SSIM (before optimization) and subjective MOS value, SSIM (after optimization) and subjective MOS value, and HSV color difference and subjective MOS value are verified using root mean square error, respectively. The steps are as follows:

[0098] Step 2.3.1 Calculate the RMSE values ​​of the PSNR and its subjective MOS value, SSIM (before optimization) and subjective MOS value, and HSV color difference and subjective MOS value of the two sample images before and after restoration obtained in Step 1 according to the following formula (4), 3 sets; and the RMSE values ​​of SSIM (after optimization) and MOS value of the two sample images before and after restoration obtained in Step 2.2.1, 1 set:

[0099]

[0100] In equation (4), N represents the sample size, s i p represents the MOS value, which represents the rating of each sample image by all users; i This represents the evaluation value of a certain objective indicator for each sample image; the RMSE values ​​of the three objective indicators PSNR, SSIM, HSV and MOS values ​​are calculated for the images before and after restoration of the two sample images respectively, and the consistency between subjective and objective indicators is compared.

[0101] Step 2.3.2 The four sets of RMSE values ​​range from [0, z), where z is related to the range of different subjective and objective evaluation indicators. The closer the RMSE is to 0, the higher the consistency between the objective indicator and the MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the RMSE value of SSIM (after optimization) is 0.3202, and the RMSE value of SSIM (before optimization) is 0.3448. The RMSE value after optimization is 0.0246 lower than that before optimization, indicating that the optimized SSIM is effective. Based on SSIM, an objective evaluation indicator that is more reliable than the other three objective indicators PSNR, SSIM and HSV in the evaluation of ancient character restoration has been generated.

[0102] As a specific step of the present invention, in step 3, the missing parts and background color of the ancient calligraphy and painting image are repaired using Photoshop application software and AI generation tools, including: supplementing the missing edges and seal parts of the ancient characters provided by the museum using Photoshop application software; and supplementing the missing background color of the ancient characters provided by the museum using AI generation tools.

[0103] As a specific step of the present invention, step 3 involves supplementing the missing edges of the ancient characters provided by the museum using Photoshop software, including the following steps:

[0104] Step 3.1 Complete the text: Place the missing sample and rubbing provided by the museum into the Photoshop canvas, duplicate the rubbing layer, adjust the rubbing's opacity and place it above the missing sample; use the free transform tool to transform the rubbing, and fill in the missing parts of the sample by changing the size of the rubbing; after each part is filled, use the lasso tool or selection tool to select the filled part of the rubbing (including the background), and create new layers for the selected parts; name the new layers, enlarge each missing part and adjust its position and details to ensure a complete connection with the missing part of the sample; merge the text layers and the chapter layers;

[0105] Step 3.2 Complete the stamp: Using the "Channels" function in Photoshop, adjust the image's levels, contrast, saturation, etc., to ensure the stamp's color matches the missing original image. Figure 1 The result is an image of the ancient characters after the missing content has been filled in.

[0106] As a specific step of the present invention, in step 3, the missing background color of the ancient characters provided by the museum is supplemented using an AI generation tool, including: importing the image of the ancient characters after the content is supplemented into the AI ​​generation tool, selecting the expansion tool, framing the part where the background color needs to be supplemented, and similarly using a brush to frame other areas of the image where the background color can be selected, so that the background color content of other parts can be automatically filled into the part that needs to be supplemented.

[0107] As a specific step of the present invention, in step 3, four objective evaluation indicators, namely PSNR, SSIM (before optimization), SSIM (after optimization), and HSV color difference, are used respectively, and the user's subjective visual evaluation MOS score of the restored image is combined to evaluate the formal restoration effect, and four sets of objective evaluation values ​​and one set of subjective evaluation MOS scores are obtained respectively for the missing original image and rubbing, the rubbing and the restored image, and the missing original image and the restored image.

[0108] The final restoration effect was evaluated by combining the user's subjective visual assessment MOS score of the restored image. The restored image was compared with the missing original image. Except for the HSV color difference, the PSNR, SSIM value before and after optimization, and subjective MOS value were all significantly improved compared with the original image, indicating that the restoration was completely effective. Specifically: the PSNR of the restored image was 6.0646 higher than that of the original image, indicating that the restored image was clearer and less distorted than the rubbing; the SSIM (before optimization) was 0.1569 higher than that of the original image, and the SSIM (after optimization) was 0.1838 higher than that of the original image, indicating that the structural similarity between the restored image and the missing original image was better than that of the rubbing; the HSV color difference was 0.0342 lower than that of the original rubbing, indicating that the background color of the restored image was closer to that of the original image than the rubbing.

[0109] As a specific step of this invention, in step 3, the PSNR and MOS values ​​of the missing original image and rubbing, the rubbing and restored image, the missing original image and restored image, the SSIM (before optimization) and MOS values, the SSIM (after optimization) and MOS values, the HSV color difference and MOS values, and the PLCC and RSME values ​​are calculated to verify the correlation and consistency between the four objective evaluation indicators and subjective evaluation in the restoration of ancient characters.

[0110] As shown above, the absolute PLCC value of SSIM (after optimization) and MOS is the largest, exceeding the absolute PLCC value of SSIM (before optimization) and MOS by 0.1363, exceeding the absolute PLCC value of PSNR and MOS by 0.2157, and exceeding the absolute PLCC value of HSV color difference and MOS by 0.4482, indicating that the correlation between SSIM (after optimization) and MOS is the best. Simultaneously, the RMSE value of SSIM (after optimization) and MOS is the smallest, 0.0095 smaller than before optimization, indicating the best consistency between the two. Furthermore, SSIM (after optimization) is the indicator closest to the subjective evaluation among several objective indicators. This also verifies that in the formal restoration evaluation results, the SSIM (after optimization) value of the restored image and the missing original image is closer to 1 than that of the restored image and the rubbing, and the rubbing and the missing original image, further proving the effectiveness of the restoration method of this invention.

[0111] Example 2

[0112] like Figure 2 As shown, the present invention further provides an evaluation system for the restoration of the aforementioned ancient Chinese calligraphy and painting images, the evaluation system comprising:

[0113] Ancient calligraphy and painting restoration module 10 uses artificial intelligence technology, AI tools or Photoshop software to restore missing ancient calligraphy and paintings;

[0114] Evaluation module 20 uses five evaluation methods to evaluate the repair results: PSNR, SSIM (before optimization), SSIM (after optimization), HSV color difference, and MOS score.

[0115] The verification module / evaluation comparison module 30 compares and verifies the above objective evaluation methods with the user's subjective visual evaluation results, providing a reference for the restoration evaluation of ancient calligraphy and paintings by museums or related art institutions.

[0116] Example 3

[0117] like Figure 3 As shown, the present invention also provides an electronic device for the evaluation method and system for the restoration of ancient Chinese paintings and calligraphy. The electronic device includes: a memory 41, a processor 40, a bus 42, and a communication interface 43. The communication interface 43 and the processor 40 are connected to the memory 41 via the bus 42. The processor 40 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0118] Example 4

[0119] The present invention further provides an evaluation method and system for the restoration of ancient Chinese paintings and calligraphy, and a computer-readable storage medium for the electronic device. The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions trigger the processor to run the evaluation method for the restoration of ancient Chinese paintings and calligraphy of the present invention.

[0120] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the invention as defined by the appended claims.

Claims

1. An evaluation method for images used in the restoration of ancient Chinese calligraphy and paintings, comprising: Step 1: The restored images of the first and second ancient paintings and calligraphy works, which were restored using artificial intelligence technology and Photoshop software, were used as preprocessing samples for evaluation. At the same time, three existing objective evaluation indicators were selected: peak signal-to-noise ratio, structural similarity index, and color difference, combined with subjective visual MOS score to evaluate the restoration results. Step 2: Based on the assessment and restoration results, optimize the weights of each feature component of one of the objective assessment indicators from Step 1, the Structural Similarity Index (SSIM), in the assessment of ancient calligraphy and painting restoration. Step 3: Use Photoshop and AI generation tools to formally restore the missing parts and background color of the ancient calligraphy and painting image. Use four objective evaluation methods before and after optimization and subjective MOS scores to evaluate the restoration results, verify the effectiveness of the restoration, and further verify the correlation and consistency between the optimization indicators and the subjective results.

2. According to the evaluation method for the restoration of ancient Chinese calligraphy and painting images as described in claim 1, in step 1, two examples are used in this preprocessing: M-grid ancient characters restored based on artificial intelligence technology and ancient characters provided by the museum restored using PS software; the images before and after restoration of both examples need to be uniformly cropped into M regions, which are marked as 0 to M-1 as numbers.

3. According to the evaluation method for the restoration of ancient Chinese paintings and calligraphy images as described in claim 1, in step 1, three objective evaluation indicators, PSNR, SSIM and HSV, are selected. Combined with the subjective visual evaluation MOS score of the user on the restored image, each region of the two examples is evaluated, and three sets of objective evaluation values ​​and one set of subjective evaluation MOS scores are obtained for each region of the two examples before and after restoration.

4. The evaluation method for the restoration of ancient Chinese paintings and calligraphy images according to claim 1, comprising the three objective evaluation indicators PSNR, SSIM, and HSV, and a set of subjective evaluation MOS scores in step 1, wherein: PSNR is used to evaluate the degree of distortion of images before and after restoration. The higher the value, the better the restoration effect. SSIM comprises three components: luminance similarity (l), contrast similarity (c), and structural similarity (s), with values ​​ranging from 0 to 1. The closer the values ​​of the images before and after restoration are to 1, the better the restoration effect. HSV is used to evaluate the difference in background color between the images before and after restoration. After processing such as computer code unit conversion, its value ranges from 0 to 1. The closer it is to 0, the smaller the color difference between the two images, and the better the restoration effect. The restoration results are subjectively scored, with scores ranging from 1 to 5, representing very dissatisfied, dissatisfied, neutral, satisfied, and very satisfied, respectively. The MOS value is the average of all user scores.

5. The evaluation method for the restoration of ancient Chinese paintings and calligraphy images according to claim 1, in step 2, optimizing the weights of each feature component of SSIM based on the restoration evaluation results of the ancient paintings and calligraphy in step 1, includes: We analyze two different examples to determine the impact of each feature component of SSIM on the evaluation in different regions and optimize the SSIM index. The correlation and consistency between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value were verified respectively. The influence of each feature component of SSIM on the evaluation was analyzed in different regions for two types of samples, and the SSIM index was optimized. The specific steps are as follows: Step 2.1.1 Extract the SSIM component values ​​l, c, and s for each region of the two types of samples before and after restoration, and compare the changing trends of l with SSIM, c with SSIM, and s with SSIM. For ancient calligraphy and painting images restored by artificial intelligence technology or Photoshop software, the influence of the three components in the SSIM evaluation is c>l>s, specifically: In the evaluation of the restored ancient characters in the M-grid image, if M=6, the SSIM values ​​of each region before and after restoration are ranked as follows: 0.6534 (number 3) > 0.6402 (number 0) > 0.6317 (number 4) > 0.6250 (number 2) > 0.6147 (number 1) > 0.552 (number 5). The l-component of the SSIM values ​​of each region before and after restoration is ranked as follows: 0.9673 (number 0) > 0.966 (number 1) > 0.9652 (number 2) > 0.9648 (number 4) > 0.9612 (number 3) > 0.9569 (number 5). The order of the c component of the SSIM values ​​before and after region restoration is 0.3851 (number 3) > 0.3777 (number 0) > 0.3704 (number 4) > 0.352 (number 2) > 0.3377 (number 1) > 0.2876 (number 5). The order of the s component of the SSIM values ​​is 10.114 (number 1) > 10.0455 (number 3) > 9.9878 (number 2) > 9.8555 (number 5) > 9.7766 (number 0) > 8.99 (number 4). At this point, c and SSIM show the same trend, then l is closer, and finally s. Step 2.1.2 Set the sum of the three components to 1. According to the weight tendency of c>l>s in Step 2.1.1, further allocate the weights. Based on the existence of the three components l, s, and c in SSIM in Step 2.1.1, the weight combinations are: 3:5:2, 4:5:1, 3:6:1, 4:6:0, 2:7:1, 3:7:0, 2:8:0, 1:9:0; Step 2.1.3 Using the gray-level co-occurrence matrix, extract the texture maps of the three features l, c, and s of each region after the two types of sample repairs according to the weights in Step 2.2.2; Step 2.1.4 Use SSIM and PSNR to evaluate the texture maps and restored images of each region in both examples; Step 2.1.5 Identify the feature combination corresponding to the optimal evaluation result for each region in the two types of samples in 2.1.4, and calculate the normalized results of the three feature components of SSIM according to the following formula (1): In equation (1): x is the l, s, c weight value corresponding to the optimal evaluation result of the Mth region of the two types of samples using SSIM and PSNR in step 2.4; S is the standard deviation of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.1.4; μ is the average value of the l, s, c weight values ​​corresponding to the optimal evaluation result of each region of the two types of samples using SSIM and PSNR in step 2.1.

4. M The value of x after normalization is the final weight of each feature in the Mth region after the two samples are calculated separately; Step 2.1.6 Substitute the weights calculated in Step 2.1.5 into the following equation (2) to calculate the optimized SSIM value: In equation (2), SSIM(X,Y) represents the SSIM value of the entire image before and after restoration, l(X,Y) represents their brightness similarity, c(X,Y) represents their contrast similarity, s(X,Y) represents their contrast similarity, M represents the image segmentation block, and when the example is a 6-grid ancient character, M=6, M is determined according to the initial number of segmentation blocks; α represents the brightness similarity weight, since brightness determines the whole, it is not calculated according to the block; β represents the contrast similarity weight; γ represents the structural similarity weight. At this time, when x in equation (1) M When the result is the result after normalization of l: α M =x M When x in equation (1) M When the result is normalized to c, it must satisfy: β1 + β2 + ... + βj = x1 + x2 + ... + x M =1, when x in equation (1) M When the result is the normalized result of s, it must satisfy γ1+γ2+···+γj=x1+x2+···+x M =1.

6. The evaluation method for restoring images of ancient Chinese paintings and calligraphy as described in claim 1, wherein step 2 verifies the correlation and consistency between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value, including: The correlation between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value were verified using Pearson correlation coefficient. The consistency between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value was verified using root mean square error.

7. The evaluation method for the restoration of ancient Chinese paintings and calligraphy images according to claim 1, in step 2, the correlation between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value are verified using Pearson correlation coefficient, as follows: Step 2.2.1 Substitute the weights of each component obtained in step 2.1.5 into the above formula (2) to calculate the optimized SSIM values ​​before and after the whole frame restoration of the two samples respectively; Step 2.2.2 Calculate the PSNR and its subjective MOS value, the SSIM and subjective MOS value before optimization, and the PLCC value of HSV color difference and subjective MOS value of the two sample images before and after restoration, obtained in Step 1, according to the following formula (3), 3 sets; and the PLCC value of optimized SSIM and subjective MOS value of the two sample images before and after restoration, obtained in Step 2.2.1, 1 set: In equation (3), N represents the sample size, s i This represents the MOS value, which represents the rating of each sample image by all users. p represents the average MOS score of all users on N sample images; i This represents the evaluation value of a certain objective indicator for each sample image. This represents the average value of an objective indicator for N sample images; calculate the PLCC values ​​of the four objective indicators and the subjective MOS value for the images before and after restoration in two different examples, and compare the correlation between the subjective and objective indicators. Step 2.2.3 The four sets of PLCC values ​​are in the range of [-1, 1]. The closer the absolute value of PLCC is to 1, the higher the correlation between the objective indicator and the subjective MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the PLCC values ​​of the four objective indicators and the subjective MOS value are all 1, and the RMSE value needs to be compared further.

8. The evaluation method for the restoration of ancient Chinese paintings and calligraphy images according to claim 1, in step 2, the consistency between PSNR and subjective MOS value, SSIM before optimization and subjective MOS value, SSIM after optimization and subjective MOS value, and HSV color difference and subjective MOS value are verified using root mean square error, as follows: Step 2.3.1 Calculate the PSNR and its subjective MOS value, the SSIM and subjective MOS value before optimization, and the RMSE value of HSV color difference and subjective MOS value of the two sample images before and after restoration obtained in Step 1 according to the following formula (4), 3 sets; and the RMSE value of optimized SSIM and subjective MOS value of the two sample images before and after restoration obtained in Step 2.2.1, 1 set: In equation (4), N represents the sample size, s i p represents the MOS value, which represents the rating of each sample image by all users; i This represents the evaluation value of a certain objective indicator for each sample image; the RMSE values ​​of the four objective indicators and the subjective MOS value are calculated for the images before and after restoration of the two sample images respectively, and the consistency between the subjective and objective indicators is compared. Step 2.3.2 The four sets of RMSE values ​​take values ​​in the range (0, z), where: z is related to the range of values ​​of different subjective and objective evaluation indicators. The closer the RMSE is to 0, the higher the consistency between the objective indicator and the subjective MOS value, and the more reliable the evaluation result of the objective indicator. At this time, the RMSE value of the optimized SSIM is 0.3202, and the RMSE value of the unoptimized SSIM is 0.3448. The optimized RMSE value is 0.0246 lower than that before optimization, indicating that the optimized SSIM is effective. Based on SSIM, an objective evaluation indicator that is more reliable than the other three objective indicators in the evaluation of ancient character restoration is generated. The evaluation preprocessing is over. Proceed to Step 3, the evaluation phase: The missing parts and background colors of ancient calligraphy and paintings were restored using Photoshop and AI generation tools. This included: using Photoshop to supplement the missing edges and seals of ancient characters provided by the museum; and using AI generation tools to supplement the missing background colors of ancient characters provided by the museum. The missing edges of ancient characters provided by the museum were supplemented using Photoshop, including the following steps: Complete the text: Place the missing sample and rubbing provided by the museum into the Photoshop canvas, duplicate the rubbing layer, adjust the rubbing's opacity and place it above the missing sample; use the free transform tool to transform the rubbing, and fill in the missing parts of the sample by changing the size of the rubbing; after each part is completed, use the lasso tool or selection tool to select the completed part of the rubbing, and create a new layer for each selected part; name the new layers, enlarge each missing part and adjust its position and details to ensure a complete connection with the missing part of the sample; merge the text layers and the chapter layers; Complete the seal: Use the "Channels" function in Photoshop to adjust the image's levels, contrast, and saturation to ensure that the seal's color matches the missing original image, thus obtaining the image of the ancient character after completing the missing content; AI generation tools are used to supplement the missing background colors of ancient characters provided by museums. This includes: importing the image of the ancient character with the supplemented content into the AI ​​generation tool, selecting the expansion tool, framing the part that needs to have its background color supplemented, and similarly using the brush to frame other areas of the image where the background color can be selected. This allows the other parts of the background color to be automatically filled into the part that needs to be supplemented. The formal restoration effect was evaluated by using four objective evaluation indicators: PSNR, before optimization, SSIM, after optimization, and HSV color difference. Combined with the user's subjective visual evaluation MOS score of the restored image, four sets of objective evaluation values ​​and one set of subjective evaluation MOS scores were obtained for the missing original image and rubbing, rubbing and restored image, and missing original image and restored image. The final restoration effect was evaluated by combining the user's subjective visual assessment MOS score of the restored image. The restored image was compared with the missing original image. Except for the HSV color difference, the PSNR, SSIM value before and after optimization, and subjective MOS value were all significantly improved compared with the original image, indicating that the restoration was completely effective. Specifically: the PSNR of the restored image was 6.0646 higher than that of the original image, indicating that the restored image was clearer and less distorted than the original; the SSIM value before optimization was 0.1569 higher than that before restoration, and the SSIM value after optimization was 0.1838 higher than that before restoration, indicating that the structural similarity between the restored image and the missing original image was better than that of the original; the HSV color difference was 0.0342 lower than that of the original image before restoration, indicating that the background color of the restored image was closer to that of the original image than the original. The PSNR and subjective MOS values ​​of the missing original image and rubbing, the rubbing and restored image, the missing original image and the restored image, the SSIM and subjective MOS values ​​before optimization, the SSIM and subjective MOS values ​​after optimization, and the PLCC and RSME values ​​of HSV color difference and subjective MOS values ​​were calculated to verify the correlation and consistency between the four objective evaluation indicators and subjective evaluation in the restoration of ancient characters. The results show that the PLCC absolute value of SSIM and MOS values ​​is the largest after optimization, which is 0.1363 higher than the PLCC absolute value of SSIM and MOS values ​​before optimization, 0.2157 higher than the PLCC absolute value of PSNR and MOS values, and 0.4482 higher than the PLCC absolute value of HSV color difference and MOS values. This indicates that the correlation between SSIM and MOS values ​​is the best after optimization. At the same time, the RMSE value of SSIM and MOS values ​​is the smallest after optimization, which is 0.0095 smaller than before optimization, indicating the best consistency between the two. Furthermore, the optimized SSIM is the indicator that is closest to the subjective evaluation among several objective indicators. This also verifies that in the results of the formal restoration evaluation, the optimized SSIM value of the restored image and the missing original image is closer to 1 than the restored image and the rubbing, and the rubbing and the missing original image, indicating that the restoration is effective.

9. An evaluation system for the evaluation method applied to the restoration of images of ancient Chinese paintings and calligraphy according to any one of claims 1 to 8, comprising: Ancient calligraphy and painting restoration module: Uses artificial intelligence technology, AI tools or Photoshop software to restore missing ancient calligraphy and paintings; Evaluation module: The restoration results are evaluated using five evaluation methods: PSNR, SSIM before optimization, SSIM after optimization, HSV color difference, and subjective MOS score. Verification / Evaluation Comparison Module: The results of the five evaluation methods in the evaluation module are compared and verified to provide a reference for the restoration evaluation of ancient calligraphy and paintings by museums or related art institutions.

10. An electronic device for evaluating the restored image of ancient Chinese calligraphy and painting, the electronic device comprising a memory and a processor, the memory storing a computer program capable of running on the processor, the processor executing the computer program to implement the various steps of the evaluation method for the restored image of ancient Chinese calligraphy and painting; And an evaluation method and evaluation system for the restoration of the ancient Chinese paintings and calligraphy, and a computer-readable storage medium for the electronic device, wherein the computer-readable storage medium stores machine-executable instructions, and the machine-executable instructions, when invoked and run by a processor, trigger the processor to run in any one of claims 1 to 8.

Images