Intelligent repairing method for old portrait photo based on GDR and electronic equipment

Abstract

The invention relates to the technical field of image processing, and provides a GDR-based intelligent repairing method for an old portrait photo and electronic equipment. The objective of the invention is to solve a plurality of technical problems existing in a portrait old photo restoration process, especially defects in the aspects of improving image resolution, restoring image colors, finely restoring facial details and the like. According to the main scheme, the method comprises the steps of 1, obtaining an original old photo data set for data preprocessing; 2, the preprocessed data set is input into a GDR network for training, the GDR network comprises a Gpen super-resolution module, a DDcolor coloring module and a CSRNet color matching module which are connected in sequence, and the three modules are connected in sequence and work cooperatively through a feature sharing mechanism; step 3, obtaining a to-be-restored old photo and inputting the to-be-restored old photo into the trained GDR network; and step 4, outputting the restored image through cascade processing of the GDR network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and provides a portrait old photo intelligent repair method based on GDR and an electronic device. BACKGROUND

[0002] With the rapid development of digital image processing technology, old photo repair gradually shifts from traditional manual repair to intelligent and automatic direction. Existing image repair techniques mainly focus on single tasks such as image denoising, missing area filling, and resolution enhancement. For example, the super-resolution method based on generative adversarial network (GAN) (such as GPEN) generates high-resolution images through adversarial training, which performs outstandingly in facial detail restoration; while the DDcolor technology focuses on adaptive coloring of black and white images, and predicts color distribution through a deep learning model. In addition, the lightweight network CSRNet realizes efficient and accurate color correction in the color correction task by introducing a global modulation feature (GMF) module. However, existing technologies are often limited to a single repair dimension, lacking end-to-end processing capability for complex degraded old photos.

[0003] For example, the "high-resolution image repair method and device" with the publication number CN1 16433520A adds a random mask to the real image for repair training, solving the edge blur and texture distortion problems existing in traditional methods in high-resolution image repair. The core lies in directly processing high-resolution input using a pre-trained repair network, avoiding network structure redesign and additional training costs. However, this method still has the following limitations:

[0004] 1. Single function: only for resolution enhancement and defect repair, without integrating key links such as coloring and toning, which cannot meet the user's demand for black and white old photo full-process repair;

[0005] 2. Lack of portrait details: although random mask is used to enhance generalization, there is no targeted optimization for facial features unique to portraits (such as facial symmetry and skin texture), which can easily lead to facial distortion or detail blur in the repaired image;

[0006] 3. Color restoration missing: the high-resolution image after repair still relies on the original input color information, and cannot realize adaptive color restoration and enhancement for severely faded or black and white photos.

[0007] In addition, there is a problem of insufficient synergy when GPEN, DDcolor and CSRNet are independently applied in existing technologies. For example, the high-resolution image generated by GPEN may lack color prior, leading to subsequent color deviation, while DDcolor may produce color overflow under low-resolution input. This indicates that a single technology cannot cope with the multiple degradation coupling problems of "resolution-color-detail" in old photo repair.

[0008] Therefore, there is an urgent need for an integrated restoration method that combines super-resolution, intelligent colorization and dynamic color adjustment, which realizes the intelligent restoration of portrait old photos from clarity improvement to natural color restoration through multi-technology collaborative optimization. SUMMARY

[0009] The present application aims to solve the technical problems existing in the restoration of portrait old photos, especially the deficiencies in improving image resolution, restoring image color and finely restoring facial details.

[0010] The present application provides a portrait old photo intelligent restoration method based on GDR, comprising the following steps:

[0011] Step (1) obtaining the original portrait old photo data set for data preprocessing;

[0012] Step (2) inputting the preprocessed data set into the GDR network for training, wherein the GDR network comprises sequentially connected Gpen super-resolution module, DDcolor colorization module and CSRNet color adjustment module;

[0013] Step (3) obtaining the old photo to be repaired and inputting it into the trained GDR network;

[0014] Step (4) outputting the repaired image through the cascade processing of the GDR network; wherein the Gpen super-resolution module comprises the linkage processing of Mapping Network and GAN, and generates super-resolution data as the input of the DDcolor module; the DDcolor colorization module is deeply fused with the Backbone feature through the Color Queries guided CDB module; the CSRNet color adjustment module uses the conditional vector to modulate the feature and optimizes the output result through the GMF module.

[0015] In the above scheme, the data preprocessing in step (1) comprises:

[0016] Step (11) obtaining the original old photo to establish an initial data set;

[0017] Step (12) performing at least two enhancement operations including flipping, cropping and adding noise on the data set;

[0018] Step (1.3) dividing the enhanced data set into a training set and a test set according to a predetermined proportion;

[0019] In the above scheme, the main details of the GDR network fusion training in step (2) include:

[0020] Step (2.1) introduces a shared feature extraction layer: a shared feature extraction module is introduced in the middle part of the Gpen super-resolution module, the DDcolor colorization module, and the CSRNet tone mapping module, so that the three modules can share the same image features when processing different tasks. This design ensures that the input features of each module are consistently processed in the network, while reducing redundant computational overhead and optimizing the efficiency of the entire repair process.

[0021] Step (2.2) introduces skip connections and residual connections: skip connections and residual connections are introduced between the Gpen super-resolution module, the DDcolor colorization module, and the CSRNet tone mapping module, respectively, to ensure that information can be effectively transmitted between different modules. Through this design, the network can better preserve important low-level features and avoid information loss during processing, improving the accuracy of image detail restoration.

[0022] Step (2.3) adopts a feature sharing and adaptation mechanism: a feature sharing and adaptation mechanism is adopted between the Gpen super-resolution, DDcolor colorization, and CSRNet tone mapping network modules to ensure that each module can work collaboratively in the same feature space. Through this mechanism, each module can adjust the input features according to its own task requirements and adapt to the requirements of the next network module, thereby avoiding feature conflicts between different networks and improving the overall repair effect.

[0023] The feature sharing and adaptation mechanism adopted in step S2.3 is implemented as follows:

[0024] According to the task characteristics of each module, a feature adapter is set up for each module:

[0025] A first linear transformation layer is set at the end of the Gpen module to project the super-resolution features to the input dimension of the Backbone of the DDcolor module;

[0026] A channel adjustment layer is set before the CDB processing unit of the DDcolor module to match the channel number of the Backbone output features and the super-resolution features passed by Gpen through convolution;

[0027] A feature fusion layer is set before the conditional vector injection layer of the CSRNet module to weight and splice the colorization features output by DDcolor and the conditional vector generated by Gpen;

[0028] Step S33: Through the cross-module gradient backpropagation mechanism, the shared feature extraction layer parameters are updated synchronously during training, so that the feature spaces of the three modules remain aligned under L2 norm constraints. The specific constraint formula is:

[0029] ||F gpen -F ddcolor || 2 <ε1 and ||F ddcolor -F csrnet || 2 <ε2

[0030] Where F gpen F ddcolor F csrnet These represent the feature matrices of the three modules in the shared layer, with ε1 and ε2 being preset threshold parameters;

[0031] Step S34: During feature transfer, an adaptive scaling factor α is applied to the GPen output features received by the DDcolor module. The calculation method is as follows:

[0032] α=σ(Conv3×3(F gpen ))

[0033] Where σ represents the Sigmoid function, and Conv3×3 is a 3×3 convolution operation that enables dynamic adjustment of feature amplitude.

[0034] In the above scheme, the Mapping Network includes: a neural network that generates conditional vectors based on feature mapping of the input image, and converts the input image features into modulation parameters used by the GAN generator through cascaded processing of dimension transformation layer and feature enhancement layer.

[0035] In the above scheme, the DDcolor coloring module performs the following processing flow:

[0036] Step (b1): Extract image hierarchical features using a Backbone network based on the U-Net architecture;

[0037] Step (b2): Generate color guiding vectors from Color Queries;

[0038] Step (b3): ​​Perform cross-attention fusion of color-guided vectors and hierarchical features in the CDB module;

[0039] Step (b4): Normalize and scale the fused features using the Feature Processing module.

[0040] In the above scheme, the CSRNet color adjustment module is configured as follows: it includes a series structure of a first convolutional layer, a ReLU activation layer, a second convolutional layer, a conditional vector injection layer, a third convolutional layer, a GMF module, and a fourth convolutional layer.

[0041] The conditional vector injection layer performs the following operations:

[0042] The conditional vector generated by the Gpen module is spliced with the output features of the second convolutional layer in the channel dimension to generate a fusion feature matrix;

[0043] The third convolutional layer extracts spatial features from the fusion feature matrix;

[0044] The GMF module generates channel attention weights through global feature statistics, and dynamically modulates the extracted features;

[0045] The fourth convolutional layer maps the modulated features to the RGB color space output.

[0046] In the above scheme, the CDB module adopts a multi-head attention mechanism to establish the spatial correspondence between the color query and the image features, and through layer-by-layer residual connection, it fuses the shallow color information and the deep semantic features, and outputs a toning feature tensor with a channel number of 256.

[0047] In the above scheme, the working process of the GMF module is as follows:

[0048] Step (c1): global average pooling is performed on the input features to obtain channel statistics;

[0049] Step (c2): calculate the channel attention weight through a two-layer fully connected network;

[0050] Step (c3): multiply the attention weight with the original feature in each channel to perform modulation.

[0051] The present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program which, when executed by the processor, implements the method.

[0052] Because the present application adopts the above technical means, it has the following beneficial effects:

[0053] The present application combines three advanced image restoration techniques based on deep learning: Gpen super-resolution, DDcolor coloring and CSRNet toning. This combination of multiple technologies makes the image restoration process more complete and efficient, not only improving the resolution of the image, but also restoring and enhancing the color, meeting the diverse needs of users. The GDR network realizes efficient and accurate image restoration through reasonable module division, reduces the consumption of training resources, and ensures high-quality output. By comprehensively applying three technologies, multi-dimensional processing of image restoration is realized, which can fully meet the market demand for high-quality and all-round image restoration.

[0054] The key innovation of the present application is how to effectively integrate Gpen super-resolution technology, DDcolor coloring technology and CSRNet color adjustment technology, so as to overcome the bottleneck problems in the traditional technology in the image restoration process.

[0055] (1) Network structure compatibility and coordination: a unified input and output structure needs to be designed, and the transmission of shared features and condition vectors is ensured to ensure the collaborative work of the network. Gpen super-resolution mainly improves the resolution of the image and restores the details, DDcolor coloring focuses on the color restoration of the image, and CSRNet color adjustment optimizes the overall color performance of the image. The three networks require different input and output formats, especially in terms of image size, color space and feature dimension. Directly connecting them together will cause input and output incompatibility, thereby affecting the entire image restoration process. In order to realize the smooth combination of the three networks, a unified input and output format needs to be designed first. The input image can be an old photo of any resolution, which is adjusted to a standard size (such as 256x256) after preprocessing. At the same time, the output of each network can be seamlessly transmitted to the next network. In order to avoid conflicts between networks, step (2.1) introduces a shared feature extraction layer, i.e. some feature extraction modules in the middle part of the three networks are shared, to ensure the consistency of the features. At the same time, by transmitting a condition vector to each network, the network is provided with image-specific context information, so that each network can perform its specific task according to this information. For example, the super-resolution image generated by Gpen can be used as a condition vector for the DDcolor network to perform color prediction, and then the final color adjustment optimization is performed by CSRNet.

[0056] (2) Rational design of task sequence and information flow: the processing order of the tasks needs to be optimized to ensure that the information relied on by each network is effectively processed by the previous network, and information loss is avoided. The task sequence of the three networks needs to be carefully designed. If the task sequence is not reasonable, it will cause the loss or inaccuracy of image information in the transmission process, thereby affecting the restoration effect. For example, if the color adjustment is performed first and then the super-resolution is improved, it will cause the color distortion of the super-resolution image, or the color restoration is not accurate due to the lack of sufficient details in the color adjustment stage. The task sequence of the three networks needs to be optimized to ensure that the input of each step is the output of the previous network after effective processing. In the input and output stage of each network, step (2.2) introduces skip connections and residual connections to maintain the transmission and preservation of the output information of each network in the processing process, to avoid information loss or degradation.

[0057] (3) Feature processing difference: By adopting the feature sharing and adaptation mechanism in step (2.3), different networks can work together in the same feature space, avoiding feature conflicts. The Gpen super-resolution and DDcolor coloring techniques process different contents at the feature level. Gpen mainly processes detail recovery and image clarity, while DDcolor focuses more on color feature estimation and recovery. In addition, CSRNet color adjustment focuses more on overall color optimization. These networks have different feature extraction methods during processing, resulting in mismatched feature dimensions and levels. In order to handle the inconsistency of feature levels between different networks, the present application adopts a multi-task learning (Multi-task Learning) strategy, allowing different networks to share some intermediate layer features. Each network can adjust its input features through a feature adaptation module to adapt to the needs of the next network. In this way, it is ensured that the three can work together in a common feature space.

[0058] Comprehensive effect

[0059] Finally, the present application can provide a complete and high-quality portrait old photo intelligent repair method, which not only improves the resolution of the image, but also restores the details, colors and light and shadow of the image, ensuring that the repaired image is real, natural and rich in levels, meeting various complex repair needs. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 Figure 1 is a GDR network architecture diagram. DETAILED DESCRIPTION

[0061] The embodiments of the present application will be described in detail below. Although the present application will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present application is not limited to only these embodiments. Rather, modifications or equivalent replacements to the present application should be covered in the scope of the claims of the present application.

[0062] In addition, in order to better illustrate the present application, numerous specific details are given in the specific embodiments below. Those skilled in the art will understand that the present application can be implemented without these specific details.

[0063] The purpose of the present application is to solve the technical problems existing in the process of restoring old portrait photos, especially in improving image resolution, restoring image color, and finely restoring facial details. Specifically, the goal is to provide a comprehensive intelligent restoration solution by combining Gpen super-resolution technology, DDcolor coloring technology, and CSRNet color adjustment technology, to ensure that the restoration effect is improved in all dimensions and meets the diverse needs of users. An end-to-end comprehensive restoration solution is provided, which not only improves the resolution of the image, but also optimizes the color performance and detail restoration, meeting more complex restoration needs.

[0064] (1) Difficulty of improving image resolution

[0065] Existing technical problems: Many old portrait photos or historical photos often have low image quality and low resolution due to their age, which cannot meet the demand for high-resolution images in modern applications. Although existing super-resolution technology can improve the resolution of images, it often cannot accurately restore details, especially in facial features and lighting effects, making it difficult to achieve ideal restoration results.

[0066] Technical solution of the present application: By using Gpen super-resolution technology and GAN network, the present application can generate high-resolution images from low-resolution images without losing details. The generative adversarial network (GAN) optimizes the generation quality of the image, especially in portrait restoration, which can restore facial features (such as eyes, mouth, hairstyle, etc.) in detail, solving the problem that existing technology cannot effectively improve the resolution while maintaining details.

[0067] (2) Black and white image coloring and color restoration problem

[0068] Existing technical problems: Color restoration is an important problem in the restoration of black and white images such as historical photos and old movies. Existing technologies mostly focus on improving the resolution of images or a specific restoration task, and fail to provide effective solutions for image color restoration and enhancement. The coloring effect of many existing methods may not be natural, and the color performance may not be realistic, especially in complex images (such as skin and clothing color restoration), often inaccurate.

[0069] Technical solution of the present application: The present application combines DDcolor coloring technology, which can accurately predict and automatically fill in color in black and white images. This technology trains a large number of color images through a deep learning model to learn the color distribution rules, thereby accurately restoring the color in black and white images. At the same time, the CSRNet color adjustment technology further optimizes the color performance of the image, which can finely adjust the color to ensure natural and realistic color, solving the common color distortion and unnatural problems in existing technology.

[0070] (3) Challenges in facial detail and light and shadow restoration

[0071] Prior art problems: For portrait image restoration, especially the restoration of facial details, the prior art often has blurred or missing cases, especially the restoration of light and shadow effects, skin texture, eyes, and mouth details is often not fine enough.

[0072] Technical solutions of the present application: The present application can accurately restore facial details and optimize light and shadow effects by embedding a StyleGAN2 decoder into a Gpen super-resolution network combined with fine-tuning technology. The generative adversarial mechanism of the GAN network ensures the fineness of image details, solving the problem of imperfect facial detail restoration in the prior art, especially in facial expressions, eye contact, and skin texture, which can provide more natural and realistic restoration effects.

[0073] (4) Multi-task joint restoration needs

[0074] Prior art problems: The prior art often solves a certain specific problem such as image resolution or color restoration, lacking a comprehensive, end-to-end solution to simultaneously improve image clarity, color, and details. Due to the lack of cooperation between different restoration needs, existing methods often cannot meet the multi-dimensional requirements of users for image quality.

[0075] Embodiments

[0076] A portrait old photo intelligent restoration method based on GDR includes the following steps:

[0077] Step 1: Prepare a portrait old photo dataset and perform relevant preprocessing on the portrait dataset;

[0078] Step 2: The processed dataset enters the GDR network for training to obtain a trained GDR network;

[0079] Step 3: Obtain an old photo that needs to be restored;

[0080] Step 4: Input the obtained old photo that needs to be restored into the trained GDR network to obtain the result of the restored portrait old photo;

[0081] The above step 1 specifically includes the following steps:

[0082] Step 1.1: Obtain a large amount of portrait old photo data to obtain an original dataset.

[0083] Step 1.2: Perform one or more data enhancement operations such as selection, flipping, cropping, and adding or subtracting noise on the original dataset to obtain an enhanced dataset.

[0084] Step 1.3: The data set after data enhancement is divided into a training set and a test set according to a certain proportion;

[0085] The above step 2 specifically includes the following steps:

[0086] GDR network construction:

[0087] Step S2.al: Introducing a shared feature extraction layer: a shared feature extraction module is introduced in the middle part of the Gpen super-resolution module, the DDcolor coloring module and the CSRNet color adjustment module, so that the three modules share the same image features when processing different tasks;

[0088] Step S2.a2: Introducing a skip connection and a residual connection: a skip connection is introduced between the Gpen module and the DDcolor module, and the super-resolution feature map is directly transmitted to the CDB processing unit of the DDcolor module;

[0089] A residual connection is introduced between the DDcolor module and the CSRNet module to fuse the color guide feature and the color adjustment feature;

[0090] Step S2.a3: Adopting a feature sharing and adaptation mechanism: a feature sharing and adaptation mechanism is adopted between the Gpen super-resolution, DDcolor coloring and CSRNet color adjustment three network modules, to ensure that each module can work cooperatively in the same feature space, and each module can adjust the input feature according to its own task requirements and adapt to the requirements of the next network module.

[0091] Network training:

[0092] Step 2.bl: The images in the data set are cropped into three-channel images with a size of 256x256 pixels to reduce the consumption of training resources;

[0093] Step 2.b2: The images in the above data are trained in the GDR network. The image data generates super-resolution data through the Mapping Network and GAN of Gpen; the data generated by GAN extracts features through the Backbone of DDcolor, and combines Color Queries and CDB to perform deep fusion on color features; the Feature Processing module of DDcolor further optimizes the features and provides them to CSRNe; CSRNet uses the condition vector and the extracted features to generate the final output results through modules such as convolution, GMF and ReLU.

[0094] The GDR network includes an Input input band recovery picture → Mapping Network module → GAN module → Backbone module → Discriminator module → CDB module, and at the same time, a Color Queries module → CDB module → Feature Processing module → Conv layer → Relu layer → Conv layer → condition vector module → conv layer → GFM layer → Conv layer → Output output picture;

[0095] The Mapping Network module is responsible for converting the input information (including noise and specific features of the image) into a condition vector through a mapping network. The mapping network performs dimension transformation and feature enhancement on the input features, providing initial conditions for the subsequent generative adversarial network (GAN), thereby ensuring more accurate and effective information transmission during the generation process.

[0096] The GAN module uses the generative adversarial network (GAN) to generate high-quality data with specific features or images. This module consists of a generator (Generator) and a discriminator (Discriminator), with the generator responsible for generating images and the discriminator used to evaluate the authenticity of the images. In this process, GAN not only effectively generates high-quality image data, but also provides valuable input for subsequent image feature extraction and processing. These generated high-quality data will be passed to the Backbone of DDcolor.

[0097] The Backbone part is responsible for extracting basic features from the data generated by the GAN module. The backbone network that extracts high-quality features for color-related tasks adopts the U-Net structure, which is outstanding in image segmentation and feature extraction due to its sharp capture ability of image details. Through this module, the system can provide high-quality feature input for subsequent color-related tasks.

[0098] Color Queries further process color features. In this module, different color vectors are input, and the system extracts color features through the guidance of color vectors and combines these color features with the backbone features. This process enables effective transmission and application of color information in the generation process.

[0099] The CDB (Color Deep Block) module is a key part for processing and fusing color features. It extracts and decodes specific color-related information by deeply fusing color queries and backbone network features, thereby generating semantic color features. This module ensures that color features are accurately processed and applied in the image restoration and generation process.

[0100] Feature Processing module further optimizes the features processed by CDB to adapt to the input format of CSRNet. By integrating and optimizing features, this module can provide high-quality input for CSRNet, ensuring the efficiency and effectiveness of subsequent tasks.

[0101] Condition Vector plays a crucial role here. It combines the condition vector to provide additional contextual information or prior knowledge for the final output, helping CSRNet more effectively extract target features and ensure more accurate detail restoration in image restoration or enhancement processes.

[0102] Conv and ReLU modules further extract input features by using convolution layers and activation functions. Convolution layers are used to extract spatial features, while ReLU activation functions introduce nonlinearity to help the network capture more complex feature relationships. The introduction of this module makes the feature extraction process more comprehensive and precise, better adapting to task requirements.

[0103] Finally, the GMF (Global Modulation Feature) module is used to dynamically adjust the weight or importance of input features. It enhances the expression ability of global features to help CSRNet better adapt to different inputs or tasks, ensuring that the model can flexibly adjust and optimize the output according to different conditions.

[0104] In summary, the invention has the following characteristics:

[0105] 1. Multi-technology fusion, comprehensive improvement of repair effect: This proposal not only improves the resolution of images, but also intelligently colors and optimizes color performance, thereby meeting users' diverse needs for image repair.

[0106] 2. More accurate and delicate portrait repair: The super-resolution technology and GAN network in the proposal perform well in detail and facial feature restoration. Through the fine-tuned StyleGAN2 decoder, it can efficiently restore facial details, especially the details of eyes, mouth, hairstyle, etc., with more delicate repair effects and more natural visual effects, avoiding the problem of detail blur or distortion in existing technologies.

[0107] 3. Integrated image coloring and toning functions: By integrating DDcolor coloring technology, this proposal can realize automatic image coloring function, accurately inferring and filling colors for black and white images. At the same time, combined with CSRNet toning technology, the color performance in the repaired image is more realistic, avoiding color distortion or over-repair problems, and improving the overall visual effect.

[0108] 4. More comprehensive repair process: The present proposal provides a complete, end-to-end repair process, from image resolution enhancement, color painting to color optimization, covering multiple levels of image repair, which can meet the multi-dimensional repair needs in complex scenarios such as historical photo repair and old movie restoration. This integrated repair process is more comprehensive and efficient than the single repair method of the prior art.

Claims

1. A GDR-based intelligent restoration method for old portrait photos, characterized in that, The method comprises the following steps: Step 1: obtaining an original portrait old photo data set for data preprocessing; Step 2: inputting the preprocessed data set into a GDR network for training, wherein the GDR network comprises sequentially connected Gpen super-resolution modules, DDcolor coloring modules and CSRNet toning modules, and the three modules are sequentially connected and work cooperatively through a feature sharing mechanism; Step 3: obtaining a to-be-repaired old photo and inputting the to-be-repaired old photo into the trained GDR network; Step 4: outputting a repaired image through cascade processing of the GDR network; wherein the Gpen super-resolution module comprises linkage processing of a Mapping Network and a GAN to generate super-resolution data as input of the DDcolor module; the DDcolor coloring module performs deep fusion of a Color Query guided CDB module and a Backbone feature; and the CSRNet toning module modulates features by using a conditional vector and optimizes output results through a GMF module.

2. The method of claim 1, wherein, The data preprocessing in step (1) comprises: Step 1.1: obtaining original old photos to establish an initial data set; Step 1.2: performing at least two enhancement operations including flipping, cropping and adding noise on the data set; Step 1.3: dividing the enhanced data set into a training set and a test set according to a predetermined proportion.

3. The method of claim 1, wherein, In the step S2, the following steps are included: Step S2.1: introducing a shared feature extraction layer: introducing a shared feature extraction module in the middle part of the Gpen super-resolution module, the DDcolor coloring module and the CSRNet toning module, so that the three modules share the same image features when processing different tasks; Step S2.2: introducing a skip connection and a residual connection: introducing a skip connection between the Gpen module and the DDcolor module to directly transmit the super-resolution feature map to the CDB processing unit of the DDcolor module; introducing a residual connection between the DDcolor module and the CSRNet module to fuse the color guide feature and the toning feature; Step S2.3: adopting a feature sharing and adaptation mechanism: adopting a feature sharing and adaptation mechanism among the Gpen super-resolution module, the DDcolor coloring module and the CSRNet toning module to ensure that the modules can work cooperatively in the same feature space, and each module can adjust the input features according to its own task requirements and adapt to the requirements of the next network module.

4. The method of claim 3, wherein, The feature sharing and adaptation mechanism in step S2.3 is implemented as follows: According to the task characteristics of each module, a feature adapter is set respectively: a first linear transformation layer is set at the end of the Gpen module to project the super-resolution feature to the input dimension of the Backbone of the DDcolor module; a channel adjustment layer is set before the CDB processing unit of the DDcolor module to match the channel number of the Backbone output feature and the super-resolution feature transmitted by the Gpen through convolution; A feature fusion layer is arranged before the conditional vector injection layer of the CSRNet module to perform weighted splicing of the coloring features output by the DDcolor and the conditional vector generated by the Gpen; Step S33: In the training process, the shared feature extraction layer parameters are synchronously updated through the cross-module gradient backpropagation mechanism, so that the feature spaces of the three modules are kept aligned under the L2 norm constraint, and the specific constraint formula is: ||F gpen -F ddcolor || 2 <ε1 and ||F ddcolor -F csrnet || 2 <ε2 where F gpen , F ddcolor , F csrnet respectively represent the feature matrix of the three modules in the shared layer, and ε1, ε2 are preset threshold parameters; Step S34: In the feature transmission process, an adaptive scaling factor a is applied to the Gpen output features received by the DDcolor module, and the calculation method is: a = σ(Conv3x3(F gpen )) Wherein, σ represents the Sigmoid function, and Conv3×3 is a 3×3 convolution operation, which realizes dynamic adjustment of the feature amplitude.

5. The method of claim 1, wherein, The MappingNetwork comprises a neural network for generating a conditional vector based on the feature mapping of an input image, which converts the input image features into modulation parameters used by the GAN generator through cascaded processing of a dimension transformation layer and a feature enhancement layer.

6. The method of claim 1, wherein, The DDcolor coloring module performs the following processing procedure: Step (b1): extracting image hierarchical features through a Backbone network based on a U-Net architecture; Step (b2): generating a color guide vector from Color Queries; Step (b3): performing cross-attention fusion of the color guide vector and the hierarchical features in the CDB module; Step (b4): normalizing and scaling the fused features through the Feature Processing module.

7. The method of claim 1, wherein, The CSRNet toning module is configured in a series structure comprising a first convolutional layer, a ReLU activation layer, a second convolutional layer, a conditional vector injection layer, a third convolutional layer, a GMF module, and a fourth convolutional layer in sequence; The conditional vector injection layer performs the following operations: The conditional vector generated by the Gpen module is spliced with the output features of the second convolutional layer in the channel dimension to generate a fused feature matrix; The third convolutional layer performs spatial feature extraction on the fused feature matrix; The GMF module generates channel attention weights through global feature statistics to dynamically modulate the extracted features; The fourth convolutional layer maps the modulated features to the RGB color space for output.

8. The method of claim 1, wherein, The CDB module uses a multi-head attention mechanism to establish the spatial correspondence between the color query and the image features, and fuses the shallow color information and the deep semantic features through layer-by-layer residual connection to output a toning feature tensor with 256 channels.

9. The method of claim 1, wherein, The working procedure of the GMF module is as follows: Step (c1): performing global average pooling on the input features to obtain channel statistics; Step (c2): calculating channel attention weights through a two-layer fully connected network; Step (c3): performing a modulation operation of multiplying the attention weights with the original features channel by channel.

10. An electronic device, comprising: The device comprises a processor and a memory, and the memory stores a computer program which, when executed by the processor, implements the method of any one of claims 1-9.

Images