A method for enhancing artistic photo images

Abstract

This invention provides a method for enhancing artistic photograph images, belonging to the field of image enhancement. This method utilizes dynamically generated multi-scale fusion coefficients and a cross-domain interaction mechanism to construct a hybrid domain attention (HDAB) module to extract frequency and spatial domain feature information from artistic photograph images, achieving the fusion and enhancement of high-frequency, low-frequency, and multi-scale features. Furthermore, through normalization and contrast stretching methods, combined with a regional saliency enhancement strategy, an adaptive contrast enhancement (ACEM) module is constructed to perform contrast enhancement on artistic photograph image features, dynamically generate weight information, and highlight target region features. This method improves the visual effects of artistic photographs, enhancing their artistic expression and detail clarity.

Description

Technical Field

[0001] The present invention belongs to the field of image enhancement, and in particular relates to an artistic photo image enhancement processing method. Background Art

[0002] With the continuous development of digital imaging technology, image enhancement, as an important image processing technology, has been widely used in photography, medical imaging, satellite remote sensing, art restoration and other fields. Especially in the processing of artistic photos, how to effectively enhance the visual effect of the image while retaining the unique style and details of the artwork has become a major challenge in the current field of image enhancement.

[0003] Traditional image enhancement methods, such as histogram equalization, contrast adjustment, and image filtering, can improve image brightness and contrast, but they often have some obvious shortcomings when processing artistic photos. First, traditional methods are prone to image color distortion. Especially in works of art, the richness and gradient of colors are very important. Any excessive enhancement or adjustment of colors may destroy the style of the original work. Second, traditional technologies usually perform image enhancement in a global manner and cannot effectively process details and local features, resulting in deficiencies in the representation of image details and textures. Especially for complex artistic photos, the loss of details and local contrast is particularly serious.

[0004] In recent years, with the development of deep learning and convolutional neural networks (CNN), image enhancement technology has made significant progress. Deep learning-based methods have effectively improved image quality by automatically learning image features, especially in detail recovery, noise removal, and contrast enhancement. However, existing deep learning methods still have some problems when processing artistic photos. Since artistic photos often contain rich details, special textures, and unique color combinations, existing deep learning methods sometimes over-smooth images or fail to fully retain the original artistic style, affecting the expressiveness and artistic sense of the image.

[0005] Therefore, how to improve the visual effects of artistic photos while retaining their original artistic style and details during the image enhancement process is a difficult problem that needs to be solved urgently in the field of image processing. Current technology has not yet been able to effectively combine the special requirements of artistic photos, and it is necessary to enhance image quality while ensuring that the artistic style and details are not distorted. Therefore, proposing an image processing method that can enhance the characteristics of artistic photos has become an important research task in image enhancement technology. Summary of the Invention

[0006] The present invention provides an artistic photo image enhancement processing method, which aims to improve the quality of artistic photo images and enhance the details and clarity of the images through deep learning models and feature extraction enhancement technology.

[0007] The present invention aims to provide an artistic photo image enhancement processing method, which specifically includes the following steps.

[0008] S1. Collect artistic photos, preprocess and annotate features, and then divide them into training set, validation set and test set in the ratio of 70%, 20% and 10% for training the artistic photo image enhancement model.

[0009] S2. Through the dynamically generated multi-scale fusion coefficients and the proposed cross-domain interaction mechanism, a hybrid domain attention HDAB module is constructed. The HDAB module is used to extract the frequency domain and spatial domain information of the artistic photo image features, perform weighted fusion of high-frequency, low-frequency and multi-scale features, and enhance the frequency domain and spatial domain features through the cross-domain interaction mechanism.

[0010] S3. Through normalization and contrast stretching methods, and by proposing a regional saliency enhancement strategy, an adaptive contrast enhancement ACEM module is constructed. The ACEM module is used to enhance the image features of artistic photos. According to the regional saliency enhancement strategy, attention weights are generated to highlight key information areas and suppress redundant information.

[0011] S4. Design content loss, style loss, and high-frequency loss, construct the loss function and training strategy module, and optimize the model parameters by adjusting the weight and learning rate of the loss function to complete the training process of the artistic photo image enhancement model.

[0012] S5. Build an artistic photo image enhancement model, including input, image embedding, mixed domain attention HDAB module, adaptive contrast enhancement ACEM module, fully connected layers and output, and use loss function and training strategy to guide the training of the artistic photo image enhancement model.

[0013] S6. Obtain an artistic photo image that requires image enhancement, input it into an artistic photo image enhancement model for processing, and output a high-resolution artistic photo image.

[0014] Preferably, in step S2, a hybrid domain attention HDAB module is constructed, specifically comprising the following steps:

[0015] Step S21: First, extract frequency domain features from the input artistic photo image feature X, X∈R H×W×C , H, W and C represent the height, width and number of channels of X respectively; the frequency domain information of the artistic photo image features is extracted using Fourier transform FFT: F freq (X) = FFT(X),

[0016] Design dynamic filter H H 、H L Separation F freq(X) high-frequency and low-frequency information, filter H H 、H L Dynamically generated by global information, the generation formula is:

[0017] H H ,H L =Sigmoid(MLP(GAP(X))),

[0018] GAP is the global average pooling, Sigmod is the activation function,

[0019] High-frequency information is expressed as:

[0020]

[0021] The low-frequency information is represented as:

[0022]

[0023] IFFT is the inverse operation of Fourier transform. High-frequency features retain the texture details and edge information in artistic photos, while low-frequency features highlight global consistency, corresponding to the overall style and color distribution in artistic photos.

[0024] Then, the separated high-frequency and low-frequency features are transformed nonlinearly using 1×1 convolution. The specific formula is:

[0025]

[0026] Conv 1×1 It is a 1×1 convolution.

[0027] Step S22: Extract spatial domain features from the artistic photo image feature X, and use convolution kernels of different sizes to extract multi-scale local features.

[0028]

[0029] Among them, Conv k represents a convolution kernel of size k×k, k∈{3,5,7}.

[0030] Step S23: Introduce the dynamically generated multi-scale fusion coefficient w k , for multi-scale features Weighted summation to generate spatial domain fusion art photo image features:

[0031]

[0032] The multi-scale local features extracted using convolution kernels of different sizes are the multi-scale fusion coefficient w k , the calculation formula is:

[0033]

[0034] in, It is a feature map The feature mean of multi-scale convolution features Calculate the global mean.

[0035] It is a feature map The characteristic standard deviation of The local variation intensity is calculated.

[0036] μ k It is a dynamic offset that represents the prior preference for a specific scale k, which is obtained from the multi-scale convolution features through nonlinear transformation. Extracted from, the formula is:

[0037]

[0038] Linear k (·) is the linear transformation operation, C is the number of channels,

[0039] ∈ is a small value, and ∈ prevents the denominator from being zero.

[0040] Step S24: Generate dynamic weights through global average pooling and cross-domain interaction to dynamically adjust the importance of frequency domain and spatial domain features. First, global average pooling GAP and MLP are used to generate global artistic photo image feature weights in the frequency domain and spatial domain, which are expressed as and

[0041]

[0042] Softmax is a normalization function;

[0043] On the basis of global weight, the interactive information between frequency domain and spatial domain is added: first, the importance weight of spatial domain is generated by frequency domain features:

[0044] W spatial =Softmax(MLP(GAP(F′ freq (X)))),

[0045] Then the importance weights of the frequency domain are generated by the spatial domain features:

[0046] W freq =Softmax(MLP(GAP(F′ spatial (X)))),

[0047] Finally, the global weight and cross-domain weight are combined to generate the final weight:

[0048]

[0049] λ1, λ2, λ3, and λ4 are learnable parameters.

[0050] Step S25: The frequency domain and spatial domain features are weightedly fused to output the artistic photo image feature Z.

[0051] Z=Weight freq ·F′ freq (X)+Weight spatial ·F′ spatial (X).

[0052] Preferably, in step S2, for the feature extraction module and the mixed domain attention HDAB module, the dynamic decomposition and fusion of frequency domain and spatial domain features, combined with multi-scale feature extraction and weight adjustment mechanism, effectively enhances the model's ability to capture global information and local details; the dynamically generated multi-scale fusion coefficient and adaptive weight mechanism further enhance the focus on key information areas, while suppressing redundant features, thereby significantly improving the visual expressiveness and detail clarity of artistic photos; in addition, the module design takes into account both computational efficiency and performance balance, and is suitable for image enhancement tasks in multiple scenarios.

[0053] Preferably, in step S3, an adaptive contrast enhancement ACEM module is constructed, specifically comprising the following steps: Step S31, feature contrast enhancement is performed, inputting the artistic photo image feature X processed by the hybrid domain attention HDAB module, X∈R H ×W×C , H, W, and C represent the height, width, and number of channels of X, respectively;

[0054] First, the input art photo image features are normalized to enhance the contrast of the art photo image features. The normalized feature map X norm A distribution with mean 0 and standard deviation 1: X norm Expressed as:

[0055]

[0056] mean(X) and std(X) represent the mean and standard deviation of the feature map X, respectively. ∈ is a small value to prevent the denominator from being zero.

[0057] Step S32: X norm Perform contrast stretching to map the feature value range to a larger dynamic range and enhance the feature contrast. Specifically:

[0058]

[0059] Xmin and X max is the feature map X norm Clip(x,a,b) means limiting the value of x to the interval [a,b]. This operation sets values ​​less than a to a and values ​​greater than b to b.

[0060] Step S33: Enhance the features of a specific region in the artistic photo image using a regional saliency enhancement strategy, making the target region more prominent.

[0061] First, the global artistic photo image feature information is extracted through average pooling and maximum pooling:

[0062] F avg =AvgPool(X enhanced ), F max =MaxPool(X enhanced ),

[0063] AvgPool means average pooling, MaxPool means maximum pooling,

[0064] The features are fused through convolution operations to generate attention weights. First, the features are fused through 7×7 convolution:

[0065] F conv =Conv 7×7 ([F avg ,F max ]),

[0066] Conv 7×7 is a 7×7 convolution, [F avg , F max ] indicates feature splicing,

[0067] Use the trigonometric functions sin and cos combined with dynamic adjustment coefficients α and β to generate attention weights:

[0068] A spatial = sin(α·F conv )+cos(β·F conv ),

[0069] Among them, α is used to dynamically adjust the frequency of sin, and β is used to dynamically adjust the frequency of cos. The calculation formulas of α and β are:

[0070]

[0071] Mean(F conv ) and Std(F conv ) represent the features F conv The mean and standard deviation of ∈ are small values, ∈ prevents the denominator from being zero,

[0072] Then for A spatial Normalize to ensure that the value range is within [0,1]. The specific formula is:

[0073]

[0074] max(A spatial ) and min(A spatial ) represent feature A spatial The maximum and minimum values ​​of spatial For the input feature X enhanced Point-by-point weighting, the specific formula is:

[0075] X spatial =X enhanced ·A spatial .

[0076] Preferably, in step S3, for the adaptive contrast enhancement ACEM module, by combining normalization and contrast stretching technology, attention weights are dynamically generated and normalized, which significantly improves the model's ability to focus on key areas of the image and suppress redundant information; at the same time, by combining average pooling and maximum pooling to extract global features, and then using dynamic trigonometric functions to generate weights, the model is given stronger expression and adaptability, which can effectively enhance the detail expression and overall visual effect of artistic photos; in addition, this method is flexible and applicable to a variety of complex scenes, taking into account the balance between local details and global consistency.

[0077] Preferably, in step S4, constructing a loss function and training strategy module specifically includes the following steps:

[0078] To ensure that the generated artistic photo images achieve the desired effect in both structure and style, we will use three key losses: Content Loss, Style Loss, and High-Frequency Loss:

[0079] The content loss is calculated using the intermediate layer features of the VGG-19 network. These layers capture the high-level structural features of the image. The calculation formula for content loss is:

[0080]

[0081] and They represent the feature maps of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, respectively. N is the total number of elements in the feature map, N = C × H × W, that is, the number of channels × height × width, ||·|| 2 represents the L2 norm;

[0082] The Gram matrix is ​​used to capture the style characteristics of the image. The Gram matrix represents the correlation between different features. The style consistency is optimized by comparing the Gram matrices of the target art image and the generated image. The calculation formula of the style loss is:

[0083]

[0084] and denote the Gram matrices of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, respectively, and M is the total number of elements in the feature map;

[0085] High-frequency loss extracts the high-frequency components of the enhanced image and the target art image and compares their differences in the high-frequency space, thereby enhancing the performance of edges and details. Specifically, the Laplacian operator is used to extract the high-frequency components (edges and textures) of the image. The calculation formula of high-frequency loss is:

[0086] L highfreq =||Laplacian(G(X))-Laplacian(X)|| 2 ,

[0087] X represents the target art image, G(X) represents the enhanced image, that is, the output after processing by the HDAB and ACEM modules; Laplacian (·) represents the Laplacian operator, which is used to extract high-frequency information (edges and textures);

[0088] In order to optimize the three objectives of content, style and high-frequency loss at the same time, the total loss function is designed:

[0089] L total =αL content +βL style +γL highfreq ,

[0090] α, β, and γ represent the weight coefficients of content loss, style loss, and high-frequency loss, respectively. At the same time, the weights are dynamically adjusted during the training process. In the early stage: α>β, mainly optimizing content loss to ensure the accuracy of the structure of the generated image; in the middle stage: gradually increase β to optimize style consistency; in the later stage: increase γ to enhance the authenticity of the image; in the early stage, emphasize content loss, and in the later stage increase the weights of style loss and high-frequency loss to optimize the overall artistry of the generated image.

[0091] Preferably, in step S4, for the loss function and training strategy module, the three parts of content loss, style loss and high-frequency loss are used comprehensively, and the model's ability to generate artistic photos is improved through joint optimization; content loss focuses on structural consistency, style loss captures the details of style features through the Gram matrix, and high-frequency loss uses the Laplace operator to enhance the ability to capture edges and details; the dynamic adjustment strategy of weight parameters further balances the content, style and detail performance of the generated image, making the optimization process more flexible and efficient, and ultimately enhancing the overall artistry and detail expression of the generated artistic photos.

[0092] Compared with the prior art, the present invention has the following technical effects:

[0093] The present invention introduces a hybrid-domain attention HDAB module to extract multi-scale features from the frequency and spatial domains, combines it with the adaptive contrast enhancement ACEM module to dynamically adjust feature contrast, and adopts a multi-objective optimization strategy of content loss, style loss, and high-frequency loss to achieve accurate expression and enhancement of the features of artistic photo images; through a dynamic weight generation and optimization mechanism, the present invention effectively highlights key information areas, suppresses redundant features, and significantly improves the structural consistency, style matching, and detail expression of the generated images. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 This is a flow chart of an artistic photo image enhancement processing method provided by the present invention.

[0095] Figure 2 This is a structural diagram of the hybrid domain attention HDAB module provided by the present invention.

[0096] Figure 3 This is a structural diagram of the adaptive contrast enhancement ACEM module provided by the present invention.

[0097] Figure 4 It is a low-resolution artistic photo image provided by the present invention.

[0098] Figure 5 The present invention provides a high-resolution rendering obtained by processing a low-resolution artistic photo image. DETAILED DESCRIPTION

[0099] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0100] Please see the attached Figure 1-5, the present invention provides an artistic photo image enhancement processing method.

[0101] As attached Figure 1 The present invention proposes a method for enhancing an artistic photo image, which includes the following steps:

[0102] S1. Collect artistic photos, preprocess and annotate features, and then divide them into training set, validation set and test set in the ratio of 70%, 20% and 10% for training the artistic photo image enhancement model.

[0103] Furthermore, in step S1, artistic photo images are collected to create a dataset to ensure that the data source has the characteristics of high resolution and no compression loss; secondly, all images are standardized and preprocessed, the resolution is uniformly adjusted to 640×640 pixels, and the existing compression artifacts or noise are removed; then, according to the artistic style and visual characteristics of the image, the target enhancement requirements are manually annotated, including texture detail preservation, color saturation enhancement and style consistency features, to generate an enhanced target image that matches each input image; finally, the dataset is divided into training set, validation set and test set in the ratio of 70%, 20% and 10% to ensure balanced data distribution within each set.

[0104] S2. Through the dynamically generated multi-scale fusion coefficients and the proposed cross-domain interaction mechanism, a hybrid domain attention HDAB module is constructed. The HDAB module is used to extract the frequency domain and spatial domain information of the artistic photo image features, perform weighted fusion of high-frequency, low-frequency and multi-scale features, and enhance the frequency domain and spatial domain features through the cross-domain interaction mechanism.

[0105] Further, in step S2, for the mixed domain attention HDAB module, see Figure 2 shown.

[0106] Step S21: First, extract frequency domain features from the input artistic photo image feature X, X∈R H×W×C , H, W, and C represent the height, width, and number of channels of X, respectively; the frequency domain information of the artistic photo image features is extracted using Fourier transform FFT:

[0107] F freq (X) = FFT(X),

[0108] Design dynamic filter H H 、H L Separation F freq (X) high-frequency and low-frequency information, filter H H 、H L Dynamically generated by global information, the generation formula is:

[0109] H H ,HL =Sigmoid(MLP(GAP(X))),

[0110] GAP is the global average pooling, Sigmod is the activation function,

[0111] High-frequency information is expressed as:

[0112]

[0113] The low-frequency information is represented as:

[0114]

[0115] IFFT is the inverse operation of Fourier transform. High-frequency features retain the texture details and edge information in artistic photos, while low-frequency features highlight global consistency, corresponding to the overall style and color distribution in artistic photos.

[0116] Then, the separated high-frequency and low-frequency features are transformed nonlinearly using 1×1 convolution. The specific formula is:

[0117]

[0118] Conv 1×1 It is a 1×1 convolution.

[0119] Step S22: Extract spatial domain features from the artistic photo image feature X, and use convolution kernels of different sizes to extract multi-scale local features.

[0120]

[0121] Among them, Conv k represents a convolution kernel of size k×k, k∈{3, 5, 7}.

[0122] Step S23: Introduce the dynamically generated multi-scale fusion coefficient w k , for multi-scale features Weighted summation to generate spatial domain fusion art photo image features:

[0123]

[0124] The multi-scale local features extracted using convolution kernels of different sizes are the multi-scale fusion coefficient w k , the calculation formula is:

[0125]

[0126] in, It is a feature map The feature mean of multi-scale convolution features Calculate the global mean.

[0127] It is a feature map The characteristic standard deviation of The local variation intensity is calculated.

[0128] μ k It is a dynamic offset that represents the prior preference for a specific scale k, which is obtained from the multi-scale convolution features through nonlinear transformation. Extracted from, the formula is:

[0129]

[0130] Linear k (·) is the linear transformation operation, C is the number of channels,

[0131] ∈ is a small value, and ∈ prevents the denominator from being zero.

[0132] Step S24: Generate dynamic weights through global average pooling and cross-domain interaction to dynamically adjust the importance of frequency domain and spatial domain features. First, global average pooling GAP and MLP are used to generate global artistic photo image feature weights in the frequency domain and spatial domain, which are expressed as and

[0133]

[0134] Softmax is a normalization function;

[0135] On the basis of global weight, the interactive information between frequency domain and spatial domain is added: first, the importance weight of spatial domain is generated by frequency domain features:

[0136] W spatial =Softmax(MLP(GAP(F′ freq (X)))),

[0137] Then the importance weights of the frequency domain are generated by the spatial domain features:

[0138] W freq =Softmax(MLP(GAP(F′ spatial (X)))),

[0139] Finally, the global weight and cross-domain weight are combined to generate the final weight:

[0140]

[0141] λ1, λ2, λ3, and λ4 are learnable parameters.

[0142] Step S25: The frequency domain and spatial domain features are weightedly fused to output the artistic photo image feature Z.

[0143] Z=Weight freq ·F′ freq (X)+Weight spatial ·F′ spatial (X).

[0144] S3. Through normalization and contrast stretching methods, and by proposing a regional saliency enhancement strategy, an adaptive contrast enhancement ACEM module is constructed. The ACEM module is used to enhance the image features of artistic photos. According to the regional saliency enhancement strategy, attention weights are generated to highlight key information areas and suppress redundant information.

[0145] Further, in step S3, for the adaptive contrast enhancement ACEM module, see Figure 3 shown.

[0146] Step S31: Perform feature contrast enhancement and input the artistic photo image feature X processed by the hybrid domain attention HDAB module, X∈R H×W×C , H, W, and C represent the height, width, and number of channels of X, respectively;

[0147] First, the input art photo image features are normalized to enhance the contrast of the art photo image features. The normalized feature map X norm A distribution with mean 0 and standard deviation 1: X norm Expressed as:

[0148]

[0149] mean(X) and std(X) represent the mean and standard deviation of the feature map X, respectively. ∈ is a small value to prevent the denominator from being zero.

[0150] Step S32: X norm Perform contrast stretching to map the feature value range to a larger dynamic range and enhance the feature contrast. Specifically:

[0151]

[0152] X min and X max is the feature map X norm Clip(x,a,b) means limiting the value of x to the interval [a,b]. This operation sets values ​​less than a to a and values ​​greater than b to b, where a=0 and b=1.

[0153] Step S33: Enhance the features of a specific region in the artistic photo image using a regional saliency enhancement strategy, making the target region more prominent.

[0154] First, the global artistic photo image feature information is extracted through average pooling and maximum pooling:

[0155] F avg =AvgPool(X enhanced ), F max =MaxPool(X enhanced ),

[0156] AvgPool means average pooling, MaxPool means maximum pooling,

[0157] The features are fused through convolution operations to generate attention weights. First, the features are fused through 7×7 convolution:

[0158] F conv =Conv 7×7 ([F avg , F max ]),

[0159] Conv 7×7 is a 7×7 convolution, [F avg , F max ] indicates feature splicing,

[0160] Use the trigonometric functions sin and cos combined with dynamic adjustment coefficients α and β to generate attention weights:

[0161] A spatial = sin(α·F conv )+cos(β·F conv ),

[0162] Among them, α is used to dynamically adjust the frequency of sin, and β is used to dynamically adjust the frequency of cos. The calculation formulas of α and β are:

[0163]

[0164] Mean(F conv ) and Std(F conv ) represent the features F conv The mean and standard deviation of ∈ are small values, ∈ prevents the denominator from being zero,

[0165] Then for A spatial Normalize to ensure that the value range is within [0,1]. The specific formula is:

[0166]

[0167] max(A spatial) and min(A spatial ) represent feature A spatial The maximum and minimum values ​​of spatial For the input feature X enhanced Point-by-point weighting, the specific formula is:

[0168] X spatial =X enhanced ·A spatial .

[0169] S4. Design content loss, style loss, and high-frequency loss, construct the loss function and training strategy module, and optimize the model parameters by adjusting the weight and learning rate of the loss function to complete the training process of the artistic photo image enhancement model.

[0170] Furthermore, in step S4, the loss function and training strategy module specifically includes the following steps:

[0171] To ensure that the generated artistic photo images achieve the desired effect in both structure and style, we will use three key losses: Content Loss, Style Loss, and High-Frequency Loss:

[0172] The content loss is calculated using the intermediate layer features of the VGG-19 network. These layers capture the high-level structural features of the image. The calculation formula for content loss is:

[0173]

[0174] and They represent the feature maps of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, respectively. N is the total number of elements in the feature map, N = C × H × W, that is, the number of channels × height × width, ||·|| 2 represents the L2 norm;

[0175] The Gram matrix is ​​used to capture the style characteristics of the image. The Gram matrix represents the correlation between different features. The style consistency is optimized by comparing the Gram matrices of the target art image and the generated image. The calculation formula of the style loss is:

[0176]

[0177] and denote the Gram matrices of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, respectively, and M is the total number of elements in the feature map;

[0178] High-frequency loss extracts the high-frequency components of the enhanced image and the target art image and compares their differences in the high-frequency space, thereby enhancing the performance of edges and details. Specifically, the Laplacian operator is used to extract the high-frequency components (edges and textures) of the image. The calculation formula of high-frequency loss is:

[0179] L highfreq =||Laplacian(G(X))-Laplacian(X)|| 2 ,

[0180] X represents the target art image, G(X) represents the enhanced image, that is, the output after processing by the HDAB and ACEM modules; Laplacian (·) represents the Laplacian operator, which is used to extract high-frequency information (edges and textures);

[0181] In order to optimize the three objectives of content, style and high-frequency loss at the same time, the total loss function is designed:

[0182] L total =αL content +βL style +γL highfreq ,

[0183] α, β, and γ represent the weight coefficients of content loss, style loss, and high-frequency loss, respectively. During the training process, the weight coefficients of content loss α, style loss β, and high-frequency loss γ are dynamically adjusted to optimize the artistic image enhancement effect. The specific adjustment strategy is as follows:

[0184] Initial stage (first 0-30% training iterations):

[0185] Set α = 10, β = 1, γ = 0, mainly optimizing content loss to ensure the accuracy of the generated image structure and avoid premature loss of original features;

[0186] Mid-stage (30%-70% training iterations):

[0187] Gradually adjust the weights to enhance the influence of style loss, setting α = 5, β = 5, and γ = 1. This enhances style consistency while maintaining the content structure, while introducing high-frequency loss to improve the quality of local details.

[0188] Late stage (70%-100% training iterations):

[0189] The weights are further adjusted to strengthen style transfer and high-frequency information, making the artistic style more prominent and the image more realistic and delicate. α=2, β=8, and γ=5 are set to reduce the impact of content loss and make the style features and local edges clearer.

[0190] S5. Build an artistic photo image enhancement model, including input, image embedding, mixed domain attention HDAB module, adaptive contrast enhancement ACEM module, fully connected layers and output, and use loss function and training strategy to guide the training of the artistic photo image enhancement model.

[0191] Further, in step S5, for the art photo image enhancement model, input the low-resolution art photo image I, I∈R H×W×C , H, W, and C represent the height, width, and number of channels of I, respectively. In this embodiment, H = 640, W = 640, C = 3, that is, the input image is I∈R 640×640×3 , input I into the image embedding, which contains 3×3 convolution, and obtain the low-resolution art photo image feature X0. Input X0 into the mixed domain attention HDAB module to obtain X1. Input X1 into the adaptive contrast enhancement ACEM module to obtain X2. Input X2 into the 3×3 convolution for processing. The processed art photo image feature is input into the fully connected layer, and finally output to obtain I + , I + ∈R 640×640×3 .

[0192] Furthermore, in step S5, the art photo image enhancement model is coded using the Pycharm application and Python language, using the Pytorch framework, and the model input is trained with a low-resolution art photo image with a resolution of 640×640×3, and the model is trained from scratch for 300 epochs.

[0193] S6. Obtain an artistic photo image that requires image enhancement, input it into an artistic photo image enhancement model for processing, and output a high-resolution artistic photo image.

[0194] Further, if Figure 4 and Figure 5 As shown, Figure 4 Shows low-resolution artistic photo images, Figure 5 The high-resolution artistic photo images processed by the artistic photo image enhancement model are displayed. It can be seen that the clarity and details of the images in the houses, trees and background parts have been significantly improved, and the details are presented more clearly.

[0195] The above are only preferred embodiments of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A method for enhancing an artistic photo image, characterized in that: The following steps are involved: S1. Collect artistic photos, pre-process and annotate them, and then divide them into training set, validation set and test set in the ratio of 70%, 20% and 10% for training artistic photo image enhancement model; S2. Using dynamically generated multi-scale fusion coefficients and the proposed cross-domain interaction mechanism, a hybrid-domain attention HDAB module is constructed. The HDAB module is used to extract frequency and spatial domain information of artistic photo image features, perform weighted fusion of high-frequency, low-frequency, and multi-scale features, and enhance the frequency and spatial domain features through the cross-domain interaction mechanism. S3. By using normalization and contrast stretching methods and proposing a regional saliency enhancement strategy, we construct an adaptive contrast enhancement (ACEM) module. We use the ACEM module to enhance the image features of artistic photos and generate attention weights based on the regional saliency enhancement strategy to highlight key information areas and suppress redundant information. S4. Design content loss, style loss, and high-frequency loss, construct loss function and training strategy modules, and optimize model parameters by adjusting the weights and learning rates of the loss functions to complete the training process of the artistic photo image enhancement model. S5. Construct an artistic photo image enhancement model, including input, image embedding, hybrid domain attention HDAB module, adaptive contrast enhancement ACEM module, fully connected layers, and output. Use loss function and training strategy to guide the training of the artistic photo image enhancement model. S6. Obtain an artistic photo image that requires image enhancement, input it into an artistic photo image enhancement model for processing, and output a high-resolution artistic photo image.

2. The method for enhancing an artistic photo image according to claim 1, wherein: In step S2, a hybrid domain attention HDAB module is constructed. The specific method is as follows: S21, first input the image features of the artistic photo Extract frequency domain features from , 、 and Respectively represent The height, width and number of channels of the image are calculated; the frequency domain information of the artistic photo image features is extracted using Fourier transform FFT: ， Designing dynamic filters 、 Separation The high-frequency and low-frequency information of the filter 、 Dynamically generated by global information, the generation formula is: ， is the global average pooling, Sigmod is the activation function, High-frequency information is expressed as: ， The low-frequency information is represented as: ， Where IFFT is the inverse operation of Fourier transform. Then, the separated high-frequency and low-frequency features are transformed nonlinearly using 1×1 convolution. The specific formula is: ， It is a 1×1 convolution; S22. From the image features of artistic photos Extract spatial domain features and use convolution kernels of different sizes to extract multi-scale local features. ， in represents a convolution kernel of size k×k, ; S23. Introducing dynamically generated multi-scale fusion coefficients , for multi-scale features Weighted summation to generate spatial domain fusion art photo image features: ， The multi-scale local features extracted using convolution kernels of different sizes are , the calculation formula is: ， in, It is a feature map The feature mean of multi-scale convolution features Calculate the global mean. It is a feature map The characteristic standard deviation of The local variation intensity is calculated. It is a dynamic offset that represents the prior preference for a specific scale k, which is obtained from the multi-scale convolution features through nonlinear transformation. Extracted from, the formula is: ， is the linear transformation operation, C is the number of channels, is a small value, Prevent the denominator from being zero; S24. Generate dynamic weights through global average pooling and cross-domain interaction to dynamically adjust the importance of frequency domain and spatial domain features. First, global average pooling GAP and MLP are used to generate global artistic photo image feature weights in frequency domain and spatial domain, which are expressed as and , ， Softmax is a normalization function; On the basis of global weight, the interactive information between frequency domain and spatial domain is added: first, the importance weight of spatial domain is generated by frequency domain features: ， Then the importance weights of the frequency domain are generated by the spatial domain features: ， Finally, the global weight and cross-domain weight are combined to generate the final weight: ， ， 、 、 、 is a learnable parameter; S25. Output artistic photo image features by weighted fusion of frequency domain and spatial domain features , 。 3. The method for enhancing artistic photo images according to claim 2, wherein: In the step S3, an adaptive contrast enhancement module ACEM is constructed, and the specific method is as follows: S31, perform feature contrast enhancement, input the artistic photo image features processed by the mixed domain attention HDAB module , , 、 and Respectively represent height, width, and number of channels; First, the input art photo image features are normalized, and the normalized feature map A distribution with mean 0 and standard deviation 1: Expressed as: ， mean(X) and std(X) represent the feature maps respectively The mean and standard deviation of is a small value, Prevent the denominator from being zero; S32, yes Perform contrast stretching to enhance the contrast of features, specifically: ， and is the feature map Clip(x,a,b) means limiting the value of x to the interval [a,b]. This operation sets the value less than a to a and the value greater than b to b. S33. Enhance the features of specific regions in artistic photos through regional saliency enhancement strategy, making the target region more prominent: First, the global artistic photo image feature information is extracted through average pooling and maximum pooling: ， ， represents average pooling, represents maximum pooling, The features are fused through convolution operations to generate attention weights. First, the features are fused through 7×7 convolution: ， is a 7×7 convolution, represents feature splicing, Use the trigonometric functions sin and cos combined with dynamic adjustment coefficients α and β to generate attention weights: ， Among them, α is used to dynamically adjust the frequency of sin, and β is used to dynamically adjust the frequency of cos. The calculation formulas of α and β are: ， ， and Represents characteristics The mean and standard deviation of is a small value, To prevent the denominator from being zero, Then Normalize to ensure that the value range is within [0,1]. The specific formula is: ， and Represents characteristics The maximum and minimum values ​​of Using attention weights For input features Point-by-point weighting, the specific formula is: 。 4. The method for enhancing an artistic photo image according to claim 3, wherein: In the step S4, the loss function and training strategy module are constructed. The specific method is as follows: The content loss is calculated using the intermediate layer features in the VGG network. These layers capture the high-level structural features of the image. The calculation formula for content loss is: ， ,and They represent the feature maps of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, is the total number of elements in the feature map, N=C×H×W, that is, the number of channels×height×width, represents the L2 norm; The Gram matrix is ​​used to capture the style characteristics of the image. The Gram matrix represents the correlation between different features. The style consistency is optimized by comparing the Gram matrices of the target art image and the generated image. The calculation formula of the style loss is: ， and Represent the Gram matrices of the target art image and the enhanced image processed by the adaptive contrast enhancement ACEM module at the i-th layer, is the total number of elements in the feature map; High-frequency loss extracts the high-frequency components of the enhanced image and the target art image and compares their differences in the high-frequency space, thereby enhancing the performance of edges and details. Specifically, the Laplacian operator is used to extract the edges and textures of the image. The calculation formula of high-frequency loss is: ， represents the target art image, represents the enhanced image, which is the output after processing by the HDAB and ACEM modules; Represents the Laplacian operator, which is used to extract the edge and texture of the image; In order to optimize the three objectives of content, style and high-frequency loss at the same time, the total loss function is designed: ， Represent the weight coefficients of content loss, style loss, and high-frequency loss respectively. At the same time, the weights are dynamically adjusted during training. In the early stage: α>β, mainly optimizing content loss to ensure the accuracy of the structure of the generated image; in the middle stage: gradually increase β to optimize style consistency; Later stage: improve γ to enhance the authenticity of the image; initially emphasize content loss, and later increase the weight of style loss and high-frequency loss to optimize the overall artistry of the generated image.

Images