Image colorization processing method, device, equipment, storage medium and program product

Abstract

The embodiment of the application provides a kind of image colorization processing method, device, equipment, storage medium and program product, it is related to image processing and artificial intelligence technical field.The method comprises: feature extraction is carried out to gray image by neural network structure, and high-order semantic feature map is generated;The neural network structure is the deep neural network that encoder and decoder are made up;Feature reconstruction is carried out to the high-order semantic feature map, and three-channel image data is generated;The three-channel image data is optimized by hybrid loss function, and target color image is generated.The method improves the authenticity, structure keeping and result stability of gray portrait color recovery, to improve the applicability of historical certificate image in identity comparison, it is advantageous to enhance the reliability of verification and improve the efficiency of financial business processing.

Description

Technical Field

[0001] This application relates to the fields of image processing and artificial intelligence technology, and in particular to an image colorization processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] As China's population ages, the proportion of elderly customers in the banking customer base is increasing year by year, and the ID photos of this group are mostly old black and white photos taken in the past. Because black and white photos lack color information, the facial recognition systems that banks rely on for transactions cannot accurately match the current facial features of customers.

[0003] Currently, in existing financial identity verification, traditional image processing methods or deep learning-based colorization methods are typically used to convert grayscale portraits into color images for historical black-and-white ID photos. However, these methods often rely on fixed mapping rules or general training data, making it difficult to balance color reproduction with facial detail representation. When dealing with portraits of elderly people or old photos that are noisy, blurry, or subject to lighting variations, color distortion and insufficient fidelity of biometric features can easily occur. Furthermore, some models have high processing overhead, making it difficult to balance recognition accuracy with real-time business requirements.

[0004] Therefore, how to improve the realism, feature fidelity, and processing efficiency of grayscale portrait colorization under complex degradation conditions has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides an image colorization processing method, apparatus, device, storage medium, and program product. Based on the colorization processing requirements of grayscale portrait images, it models the relationship between semantic information and image content in grayscale images. While taking into account the overall color generation capability of the image, it improves the expression of details in the portrait area and the stability of the result, thereby achieving effective conversion of grayscale portraits to target color images under complex degradation conditions.

[0006] In a first aspect, embodiments of this application provide an image colorization processing method, including:

[0007] Features are extracted from grayscale images using a neural network structure to generate high-order semantic feature maps; the neural network structure is a deep neural network composed of an encoder and a decoder.

[0008] Feature reconstruction is performed on the high-order semantic feature map to generate three-channel image data;

[0009] The target color image is generated by optimizing the three-channel image data using a hybrid loss function.

[0010] In one possible embodiment, feature reconstruction is performed on the high-order semantic feature map to generate three-channel image data, including:

[0011] By using skip connections, the feature maps of the corresponding layers in the encoder network and the feature maps of the decoder network are concatenated through channels to generate a fused feature map.

[0012] The fused feature map is upsampled by transposed convolution to generate three-channel image data.

[0013] In one possible embodiment, after upsampling the fused feature map via transposed convolution, the process includes:

[0014] Feature mask images of target regions are extracted using a semantic segmentation network;

[0015] The feature mask image is multiplied element-wise with the three-channel image data to generate the enhanced target color image.

[0016] In one possible embodiment, the three-channel image data is optimized using a hybrid loss function to generate a target color image, including:

[0017] The pixel distribution difference between the three-channel image data and the target color image is calculated using the cross-entropy loss function;

[0018] The local feature similarity between the three-channel image data and the target color image is calculated based on the output of the preset feature extraction network using the Dice loss function.

[0019] The pixel distribution differences and local feature similarity are weighted and summed to generate the optimized target color image.

[0020] In one possible embodiment, prior to feature extraction from the grayscale image using a neural network structure, the process includes:

[0021] Perform data augmentation on a grayscale image to generate an enhanced grayscale image;

[0022] Data augmentation specifically includes at least one of the following:

[0023] Gaussian noise injection, motion blur simulation, and brightness jitter adjustment.

[0024] In one possible embodiment, data augmentation processing is performed on the grayscale image to generate an enhanced grayscale image, including:

[0025] By adjusting the brightness and saturation of grayscale images using the HLS color space, diverse enhanced images can be generated. The HLS color space is a color representation method that includes brightness, hue, and saturation.

[0026] In one possible embodiment, feature extraction of a grayscale image is performed using a neural network structure, including:

[0027] Feature extraction from grayscale images is performed using depthwise separable convolution to generate high-order semantic feature maps.

[0028] In one possible embodiment, the method further includes:

[0029] The target color image is generated by optimizing the three-channel image data by constraining the illumination invariance.

[0030] Secondly, embodiments of this application provide an image colorization processing apparatus, comprising:

[0031] The extraction module is used to extract features from grayscale images through a neural network structure to generate high-order semantic feature maps; the neural network structure is a deep neural network composed of an encoder and a decoder.

[0032] The reconstruction module is used to reconstruct features from high-order semantic feature maps and generate three-channel image data.

[0033] The optimization module is used to optimize the three-channel image data using a hybrid loss function to generate the target color image.

[0034] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0035] The memory stores computer-executed instructions;

[0036] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method described in any of the first aspects above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in any of the first aspects above.

[0038] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0039] This application provides an image colorization processing method, apparatus, device, storage medium, and program product. It extracts features from grayscale images using a neural network structure and generates high-order semantic feature maps, enabling the extraction of deep semantic information related to colorization from grayscale portraits. By reconstructing features from the high-order semantic feature maps and generating three-channel image data, it achieves an effective mapping from grayscale representation to color representation. Furthermore, by optimizing the three-channel image data using a hybrid loss function, it balances the color reproduction effect of the color image with the fidelity of facial details, improving the colorization realism and processing effect under complex degradation conditions such as noise, blur, and changes in illumination. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 A flowchart illustrating an image colorization processing method provided in this application;

[0042] Figure 2 A flowchart illustrating an image colorization processing method based on an encoder and decoder provided in this application;

[0043] Figure 3 This application provides a schematic diagram of the structure of an image colorization processing device;

[0044] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application.

[0045] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0049] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0050] The following is a brief explanation of the terms that may be used in this application:

[0051] An encoder is a type of deep neural network module that extracts image features step by step through convolution and pooling operations.

[0052] Convolutional layer: refers to a layer that extracts local features of an image by calculating a weighted sum of local regions through a sliding window.

[0053] Pooling Layer: This refers to reducing the spatial size of the feature map by downsampling (such as max pooling) while retaining the main features.

[0054] ReLU activation function: refers to a non-linear activation function, with the formula f(x)=max(0,x), used to introduce non-linear characteristics.

[0055] Decoder: refers to a deep neural network module that gradually recovers the spatial size of an image through upsampling and feature fusion.

[0056] Transposed Convolution: This refers to an upsampling operation that calculates gradients through backpropagation to restore image resolution.

[0057] Skip Connection: This refers to directly passing encoder features to the corresponding level of the decoder, preserving low-level detailed information.

[0058] The tanh activation function is a non-linear activation function used to constrain the output range.

[0059] In scenarios where financial institutions digitize historical identification documents, verify identities, and control business risks, it is necessary to access customer-reserved photographs and verify their consistency with facial images collected on-site. For older customers, many of the earlier identification photos are black and white or grayscale scans. Typically, these grayscale images need to be colorized before being input into the subsequent recognition module.

[0060] Currently, existing image colorization methods mainly rely on image processing techniques (such as histogram equalization, color transfer, etc.) or physical models (such as illumination reflection models) to infer color information. For example, color mapping is performed by analyzing the brightness distribution of grayscale images and combining it with a prior color knowledge base (such as skin color, clothing color, etc.).

[0061] However, existing image colorization processing methods rely on fixed mapping rules or general training data, making it difficult to balance color reproduction and facial detail representation. When there is noise, blur, or changes in lighting in portraits of elderly people and old photos, color distortion and insufficient fidelity of biometric features are likely to occur. In addition, some models have high processing overhead, making it difficult to balance recognition accuracy and business real-time performance.

[0062] In view of the above problems, this application provides a method for colorizing grayscale images by using a deep neural network architecture of encoder and decoder, combined with a dynamic feature fusion mechanism and a hybrid loss function. This method can improve the authenticity, structural preservation and result stability of grayscale portrait color restoration, thereby improving the applicability of historical document images in identity comparison, which is conducive to enhancing verification reliability and improving the efficiency of financial business processing.

[0063] The entity executing this image colorization processing method can be, for example, a processing system. Optionally, the processing system can be any existing electronic device with processing capabilities, such as a terminal or a server. In some embodiments, the processing system can also be deployed in a server cluster or cloud environment. This application does not limit the deployment environment of the processing system.

[0064] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments.

[0065] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0066] Figure 1 This is a flowchart illustrating an image colorization processing method provided in this application. Figure 1 As shown, the method includes:

[0067] S101 extracts features from grayscale images using a neural network structure to generate high-order semantic feature maps.

[0068] For example, a grayscale image can be the input object to be processed, such as a historical black and white ID photo, a grayscale scan, or single-channel image data converted from a color image.

[0069] For example, the neural network structure can be a deep neural network that extracts features from a grayscale image and generates a high-order semantic feature map, and can consist of an encoder and a decoder, where the encoder is responsible for extracting image features step by step.

[0070] For example, a high-order semantic feature map can be an abstract feature information that carries the grayscale image after feature extraction. For example, a high-order semantic feature map can be a multi-channel feature expression formed after multiple convolutions and downsampling, which retains semantic information that has an indicative role in colorization, such as facial contours, facial feature positions, wrinkle textures, hair area distribution, and background structure.

[0071] Optionally, in business scenarios such as accessing historical image archives of banks, preprocessing facial recognition, and supplementing document information, the processing system can acquire grayscale images based on the image archive system, customer information management system, or front-end acceptance terminal.

[0072] Optionally, the processing system can receive the input grayscale image and perform size normalization, pixel value normalization, and human figure region alignment processing on the grayscale image.

[0073] Optionally, size normalization can uniformly adjust the input image to 256. 256 pixels, 512 A fixed resolution of 512 pixels or the same as the training configuration is used; pixel value normalization maps the original grayscale values ​​in the range of 0 to 255 to the range of 0 to 1 or -1 to 1, reducing the impact of brightness scale differences between images from different sources; portrait region alignment can be achieved by performing affine transformations on the positions of the eyes, nose tip, and corners of the mouth through facial landmark localization, improving the ability to focus on identity-related information during the feature extraction stage. After preprocessing, the single-channel grayscale image is input into the encoder.

[0074] Optionally, the encoder can employ a four-level convolutional pooling structure, with each level sequentially including convolution operations, nonlinear activation, and pooling downsampling. The convolutional layers can employ 3... A configuration with 3 convolutional kernels, a stride of 1, and identical edge padding is used to extract local texture features while preserving the spatial boundary information of the feature map; the pooling layer can use 2... A max-pooling kernel with a stride of 2 is used to progressively compress the feature map spatial size and expand the receptive field. As the network depth increases, the number of feature map channels can be gradually increased from 64 to 128, 256, and 512 to enhance the ability to express abstract semantics.

[0075] Optionally, the convolutional units in the encoder can also employ a depthwise separable convolutional structure, decomposing standard convolution into channel-wise convolution and pointwise convolution. Batch normalization layers or layer-wise normalization layers can also be introduced after the convolutional layers, along with rectified linear unit (ReLU) activation functions, leaky rectified linear unit (Leaky ReLU) activation functions, or other nonlinear mapping methods, to improve the consistency of feature distribution.

[0076] S102, perform feature reconstruction on the high-order semantic feature map to generate three-channel image data.

[0077] For example, the three-channel image data can be an intermediate result output by the decoder, specifically red-green-blue (RGB) image data, corresponding to the red channel, green channel and blue channel respectively, used to express the color components of each pixel in the color image.

[0078] For example, the decoder can be the reconstruction part of a neural network structure, which can gradually restore the low-resolution, high-semantic features formed during the encoding stage into an image representation that matches the original input size.

[0079] Optionally, the decoder may include a four-level upsampling reconstruction structure, where each level uses transposed convolutions to restore the resolution of the feature maps. For example, the transposed convolutions may use 2... Upsampling is performed using a 2-kernel convolution with a stride of 2, so that the feature map size is gradually expanded layer by layer to the same or corresponding spatial resolution as the input grayscale image.

[0080] Optionally, the processing system can also introduce a skip connection mechanism during the decoding process, which passes the feature map of the corresponding level in the encoder to the corresponding level in the decoder and performs channel concatenation with the currently upsampled feature map.

[0081] Optionally, after a certain level of decoder completes transposed convolutional upsampling, the processing system can concatenate the feature map output by the corresponding coding layer with the current feature map according to the channel dimension, and then pass the concatenation through 1 1. Convolution adjusts the number of feature channels to ensure that the channel size meets the requirements of subsequent convolutional reconstruction.

[0082] Optionally, the processing system can be in 1 A 3×3 convolution is applied after a single convolution to refine the fused features, resulting in more stable texture continuity in the generated results across hair edges, facial contours, collar areas, and background transition regions. The processing system can also incorporate an attention mechanism during the decoding stage, assigning higher weights to large areas such as the eye area, bridge of the nose, lips, and skin tone. This concentrates feature reconstruction on identity-related regions, improving the adaptability of the colorized results to subsequent face comparison tasks.

[0083] Optionally, the processing system can be based on a decoder, via 1 1. Convolution maps the feature map into three-channel image data and constrains the output value range through an activation function. For example, the Tanh function can be used to limit the output to the range of -1 to 1. Then, the numerical mapping is completed according to the image storage format to generate RGB image data that meets the requirements of display and calculation.

[0084] S103 optimizes the three-channel image data using a hybrid loss function to generate the target color image.

[0085] For example, the hybrid loss function can be trained and optimized on three-channel image data to make the generated result closer to the target color image while taking into account the consistency of color distribution and local features.

[0086] For example, the hybrid loss function can be a weighted combination of the cross-entropy loss function and the Dice loss function, where the cross-entropy loss function is used to measure the difference in pixel distribution between the generated result and the reference color image, and the Dice loss function is used to measure the similarity of local structures based on the feature space, so that the optimization process takes into account both global color restoration and key region feature fidelity.

[0087] For example, the target color image can be the output result for subsequent face recognition, authentication, image archiving, and business review.

[0088] Optionally, the processing system can construct training sample pairs, where the input to the training sample pairs is a grayscale image, and the supervised target is the corresponding real color image. The error is calculated by mixing the above three-channel image data with the target color image into a loss function.

[0089] Optionally, the cross-entropy loss function can statistically analyze the probability distribution deviation between the generated image and the reference image by pixel or by color category. In some implementations, if the color space is discretized into multiple color intervals, the cross-entropy loss can be expressed as the negative log-likelihood between the predicted color probability distribution and the true color label distribution; if a continuous color regression framework is used, the idea of ​​cross-entropy can be applied to the discretized color channels to improve the stability of color prediction.

[0090] Optionally, the Dice loss function can be calculated based on the feature maps output by a pre-defined feature extraction network. For example, the processing system can input the three-channel image data and the target color image into the pre-defined feature extraction network to obtain the feature representations of the corresponding layers. Then, the Dice coefficients are calculated based on the overlap between the two feature maps, and the Dice loss is obtained from this.

[0091] Optionally, the hybrid loss function can be expressed as: Multiply by cross-entropy loss and Multiply by the sum of Dice losses, where and These are weighting coefficients, which can be set according to the characteristics of the training data, such as 0.7 and 0.3, 0.6 and 0.4, or other ratios that meet the requirements of the task.

[0092] Optionally, if the business scenario focuses more on the consistency between the ID photo and the face comparison on site, the processing system can enhance the weight of the Dice loss; if the business scenario focuses on the visual naturalness and overall color authenticity of the archived image, the processing system can increase the weight of the cross-entropy loss.

[0093] Optionally, during training, the processing system can backpropagate and update the parameters of the neural network structure based on the error calculated by the hybrid loss function, iterating continuously until the loss converges or the preset number of training rounds is met. After the model training is complete, only a grayscale image needs to be input during the inference phase, and the target color image is output. The processing system can store the target color image in the customer's image archive or send it to the face comparison module for feature extraction and similarity calculation with face images collected on-site.

[0094] In this embodiment, the encoder abstracts and represents the facial contours, facial features, and texture information in the grayscale image step by step. The decoder restores the spatial resolution and completes the color mapping by combining skip connections. Then, the hybrid loss function simultaneously constrains the consistency between the global color distribution and local identity features. This can solve the problems of missing color information, insufficient local detail fidelity, and poor stability under complex lighting conditions when historical grayscale portraits are directly used for face recognition. This ensures that the output target color image not only meets the requirements of image colorization processing but also adapts to the subsequent identity verification business link.

[0095] It should be understood that the above examples are merely illustrative and not limiting. In one possible embodiment, the number of network layers, convolution parameters, loss weights, and input / output resolutions can be adjusted according to specific business scenarios, hardware resources, and sample characteristics, but this does not affect the basic implementation principle of the embodiments of this application.

[0096] As one possible implementation, the processing system can use skip connections to concatenate the feature maps of corresponding layers in the encoder network with the feature maps of the decoder network to generate a fused feature map. Then, the fused feature map is upsampled through transposed convolution to generate three-channel image data.

[0097] For example, an encoder network can perform stepwise feature extraction on a grayscale image to output feature maps at different levels.

[0098] For example, the decoder network can receive high-order semantic feature maps and perform feature recovery.

[0099] For example, skip connections can directly introduce feature maps of the corresponding layers in the encoder network into the decoder to preserve edge contours, texture details, and local structural information.

[0100] For example, channel stitching can be used to merge two features along the channel dimension, so that the fused feature map contains both high-order semantic information and low-level detailed information, thereby providing a richer feature base for subsequent reconstruction.

[0101] For example, transposed convolution can perform scale recovery on fused feature maps, and the convolution kernel used can be 2^32. 2. The step size can be 2, so as to output three-channel image data while expanding the spatial resolution; in practical applications, other models of this component can also be selected, and this application embodiment does not limit this.

[0102] Optionally, after receiving the high-order semantic feature map from the decoder network, the processing system aligns the decoded features of the current layer with the output of the corresponding layer of the encoder. Then, it concatenates them in the channel dimension to form a fused feature map. The system then performs a transpose convolution operation on the fused feature map to restore the feature map to the same spatial size as the target image step by step, and adjusts the number of channels to three.

[0103] The above methods introduce shallow features through skip connections, which can alleviate the lack of detail caused by relying solely on high-order semantic information; enhance feature expression capabilities through channel concatenation, so that semantic information and local texture information can participate in reconstruction in the same tensor; and complete upsampling through transposed convolution, which can form a three-channel output while restoring image resolution, thereby making the reconstruction results more suitable for subsequent face comparison.

[0104] In one possible implementation, after upsampling the fused feature map through transposed convolution, the processing system can extract a feature mask map of the target region using a semantic segmentation network. Then, the feature mask map is multiplied element-wise with the three-channel image data to generate an enhanced target color image.

[0105] For example, a semantic segmentation network can perform region recognition on image features upsampled by transposed convolution and output a feature mask map with the same spatial size as the three-channel image data.

[0106] For example, a feature mask map can characterize the spatial distribution of a target region, where the target region may correspond to the face, contour, or other preset key regions in a portrait, and the mask value is used to indicate the preservation intensity of the region in subsequent calculations.

[0107] Optionally, after obtaining the feature mask image, the processing system can perform element-wise multiplication with the three-channel image data at the same coordinate position to obtain the enhanced target color image.

[0108] By employing the above method, spatial constraints are applied to key regions through semantic segmentation networks, and region-level enhancement is achieved by combining element-wise multiplication. This improves the color consistency of key biometric features (such as lip color and pupil color), further optimizing the matching success rate of the face recognition system.

[0109] The following section provides a detailed explanation of how the assessment system generates risk feature vectors through dynamic weight allocation.

[0110] In one possible implementation, the processing system can calculate the pixel distribution difference between the three-channel image data and the target color image using a cross-entropy loss function. Then, it calculates the local feature similarity between the three-channel image data and the target color image based on the output of a pre-defined feature extraction network using a Dice loss function. Finally, it performs a weighted summation of the pixel distribution difference and the local feature similarity to generate the optimized target color image.

[0111] For example, the cross-entropy loss function can be the degree of deviation between the generated image and the target color image in the pixel-level probability distribution, and its calculation result can reflect the accuracy of the overall color reproduction.

[0112] For example, the Dice loss function can be the degree of overlap between the two in local structure and key regions. The preset feature extraction network is used to extract feature maps corresponding to the human portrait contour, skin color region and edge texture, so as to complete the similarity constraint in the feature space.

[0113] Optionally, the processing system can input the aforementioned three-channel image data into the loss calculation module. The generated image and the target color image are then pixel-normalized using the cross-entropy loss function, and a first loss value is calculated based on the distribution differences at corresponding pixel positions.

[0114] Optionally, the Dice loss function inputs the generated image and the target color image into a preset feature extraction network to obtain corresponding feature maps. Then, it measures the degree of overlap of the target regions in the feature maps to obtain a second loss value. Subsequently, the first loss value and the second loss value are multiplied by preset weight parameters and summed to obtain a mixed loss value.

[0115] Optionally, the processing system can perform reverse updates on the image reconstruction network based on the mixed loss value, and output an optimized target color image. The preset feature extraction network can adopt a convolutional neural network structure, and its parameters are preset and fixed during the training phase to improve the consistency of feature extraction. In practical applications, other models can also be selected for this network, and this application embodiment does not limit this.

[0116] Using the methods described above, the cross-entropy loss function optimizes global color accuracy through pixel-level distribution differences, while the Dice loss function calculates local feature similarity based on the output of a pre-defined feature extraction network, ensuring consistency of key biometric features. Weighted summation combines the optimization objectives of the two loss functions to generate the target color image, balancing color reproduction and biometric feature fidelity through a multi-objective optimization strategy.

[0117] In one possible implementation, before feature extraction from the grayscale image using a neural network structure, the processing system can perform data augmentation on the grayscale image to generate an enhanced grayscale image. This data augmentation includes at least one of the following: Gaussian noise injection, motion blur simulation, and brightness jitter adjustment.

[0118] For example, data augmentation can construct input samples with degraded features before grayscale images enter the neural network structure. The augmented grayscale images may contain noise, blur, or brightness variation information to improve the adaptability of subsequent feature extraction to historical ID photos, scanned images, and images with unstable shooting quality.

[0119] For example, Gaussian noise injection can be achieved by superimposing random perturbations that follow a Gaussian distribution onto the pixel values ​​of a grayscale image to simulate the graininess, scanning noise, or transmission interference of old photographs.

[0120] For example, motion blur simulation can be achieved by filtering a grayscale image using a convolutional blur kernel to characterize blur degradation caused by shooting shake, moving exposure, or imaging instability.

[0121] For example, brightness jitter adjustment can linearly scale or shift the pixel grayscale values ​​of a grayscale image to create enhanced samples at different brightness levels.

[0122] Optionally, the above three processes can be executed individually or in combination as needed for training. In practical applications, the corresponding noise intensity, blur kernel size, and brightness variation range can be set according to the sample quality and model training objectives. This application embodiment does not limit this.

[0123] Optionally, the processing system can read grayscale images from an archive image library, an identity verification platform, or a local storage unit. Then, the preprocessing module performs enhancement operations on the images according to the training configuration and inputs the processed and enhanced grayscale images into the subsequent neural network.

[0124] Through the above processing, the training samples can cover a richer range of historical image degradation patterns, enabling the model to maintain strong feature extraction robustness even when faced with old ID photos, low-resolution scans, or grayscale portraits captured under poor on-site conditions. This reduces feature shifts caused by noise, blur, or lighting differences, improves the consistency between the colorization results and the original portrait structure, and further enhances the accuracy and stability of subsequent identity verification and business verification.

[0125] By using the above methods, data augmentation processing simulates typical degradation factors of historical photos (such as noise and blur), improves the model's adaptability to complex scenes, enhances the model's robustness to degradation factors such as uneven lighting and noise, and further reduces the recognition failure rate caused by image quality degradation.

[0126] In one possible implementation, the processing system can adjust the brightness and saturation of a grayscale image using the Hue LightnessSaturation (HLS) color space to generate diverse enhanced images. The HLS color space is a color representation method that includes hue, brightness, and saturation.

[0127] For example, the HLS color space can express the brightness, hue, and saturation information of an image separately, where brightness is used to characterize the lightness or darkness of the image, saturation is used to characterize the color purity, and hue is used to characterize the color category.

[0128] Optionally, for grayscale images, since their original color information is missing, the processing system can map grayscale pixels to the HLS representation domain, and then perform parameter perturbation around the luminance and saturation components to form multiple appearance difference samples without changing the subject outline and facial structure relationship.

[0129] Optionally, the processing system can set multiple adjustment coefficients for the luminance component and multiple perturbation ranges for the saturation component according to a preset sampling strategy, so that the same input grayscale image can generate enhanced images with different light intensities and different color representations.

[0130] Optionally, the adjustment coefficients can be determined based on the statistical distribution of the training set or empirical thresholds. For example, brightness can be increased or decreased within a set range to simulate dark, normal, and bright acquisition scenarios; saturation can be varied within a low to medium range to avoid the enhancement result deviating from the real appearance of the portrait. The enhanced image obtained after this processing can be used as a training sample to input into subsequent neural network structures to improve the model's adaptability to changes in lighting, scanning differences, and fluctuations in the quality of archival images.

[0131] By using the above method, diverse lighting samples are generated through HLS color space adjustment, allowing the model to be exposed to various lighting conditions during the training phase. This improves the model's adaptability to complex lighting scenarios such as strong light and weak light, and further optimizes the color stability of the generated images under lighting changes.

[0132] As one possible implementation, the processing system can extract features from grayscale images using depthwise separable convolutions to generate high-order semantic feature maps.

[0133] For example, depthwise separable convolution can be an operation that decomposes standard convolution into depthwise convolution and pointwise convolution.

[0134] Optionally, depthwise convolution can be used to extract spatial features along the channel dimension, while pointwise convolution can be used to fuse features from different channels.

[0135] Optionally, the processing system can also adjust the convolution kernel size, number of channels, and number of layers according to the input image resolution to adapt to grayscale portrait images with different acquisition quality and file sources; in practical applications, other models of this component can also be selected, and this application embodiment does not limit this.

[0136] In this way, the original pixel information in the grayscale image is transformed into a more discriminative high-order semantic feature map. This reduces the computational complexity of the network and enhances the ability to represent key regions and detailed features of the portrait, enabling the subsequent reconstruction process to more stably recover three-channel image data that conforms to the target distribution. Therefore, the accuracy, processing efficiency, and adaptability to complex archival images can be improved in grayscale portrait colorization processing, providing a more reliable image foundation for subsequent identity verification.

[0137] By using the above methods, the computational complexity of the model is reduced through depthwise separable convolution, the efficiency of image processing is improved, the processing time of a single image meets the real-time requirements, and the cost of manual intervention in banking operations is further reduced.

[0138] Furthermore, the processing system can also optimize the three-channel image data through illumination invariance constraints to generate the target color image.

[0139] For example, illumination invariance constraints can be optimized by comparing the generated results of the same image under different illumination conditions.

[0140] Optionally, after obtaining the three-channel image data, the processing system can construct corresponding input samples under different lighting perturbations and perform feature alignment on the generated results of the same image under strong light, weak light, or color temperature shift conditions. The processing system can calculate pixel-level differences, color statistical distribution differences, and deep feature representation differences, and map the differences into constraint terms added to the optimization objective function to correct the pixel distribution and local structure of the three-channel image data and constrain the output consistency under different lighting conditions.

[0141] Optionally, the illumination invariance constraint can be applied to colorization scenarios involving historical bank ID photos, scanned documents, and images acquired in low light conditions. Since the same customer image may originate from different eras, different scanning devices, or different shooting environments, the image brightness and contrast can vary significantly. Without an illumination consistency constraint, the colorization result is prone to skin tone shift, background color cast, or local color distortion. By introducing this constraint, the generated result can maintain high color stability under complex lighting conditions, thereby improving the reliability of subsequent face recognition modules in matching the target image.

[0142] By using the above method, the illumination adaptability of the generated image is optimized through illumination invariance constraints, which reduces the recognition error caused by illumination differences, improves the usability of historical facial data in identity verification, and enhances the adaptability of colorization results in financial business processing scenarios.

[0143] Figure 2 This is a flowchart illustrating an image colorization processing method based on an encoder and decoder provided in this application. Figure 2 As shown, the method includes:

[0144] Step 1: Input and encoder downsample layer by layer;

[0145] The input image is fed into the encoder, which first performs two consecutive 3D encoding operations. 3 convolutions combined with ReLU activation yield a feature map with 64 channels; subsequently, 2 2. Max pooling completes downsampling; repeat twice. 3. 3. ReLU convolution operation, feature channels updated to 128; repeat the above two 3 operations in sequence. 3 convolutions, ReLU activation, 2 2. Max pooling operation, the feature channels are gradually changed to 256, 512, and 1024, to gradually complete the multi-scale deep feature extraction.

[0146] Step 2: Decoder upsampling and splicing with skip-layer features;

[0147] The encoder end 1024 channel features are characterized by 2 2. Upsampling convolution is used for upsampling, and simultaneously, shallow features from the corresponding layer of the encoder are retrieved through copied and cropped skip connections, and feature concatenation is completed; the concatenated features are then processed twice in sequence. 3. Convolution and ReLU activation, repeatedly perform upsampling convolution, skip concatenation, and double convolution, with the feature channels gradually decreasing from 512 to 128.

[0148] Step 3: End convolution and image output;

[0149] When the feature channels are reduced to 64, use 1 1. Convolution performs mapping transformation on the feature channels, and finally outputs the segmented target image.

[0150] Optionally, the encoder extracts image features and consists of a series of convolutional and pooling layers (or downsampling layers). Each pooling operation reduces the spatial size of the feature map and gradually extracts higher-level abstract features. A four-level downsampling structure is used to extract multi-level features through convolutional layers. For example, with an input of 224... A 224 grayscale image is generated by convolutions with 64 channels, 128 channels, 256 channels, and 512 channels, respectively.

[0151] For example, the different parameter types, configurations, and corresponding effects of the encoder can be shown in Table 1 below:

[0152] Table 1

[0153]

[0154] The kernel size can be configured to 3 3. Used for extracting local features; the convolution stride can be configured to 1 to control the feature size variation. The number of channels output by the encoder can be 64, 128, 256, and 512.

[0155] Optionally, the decoder can upsample the features extracted by the decoder and fuse them with lower-level features through skip connections to gradually recover the spatial size and detail information of the image. Skip connections are the process of directly connecting the output of a layer in the encoder to the corresponding layer in the decoder. In the encoder-decoder structure, such connections can transmit finer edge information and allow the network to directly access these features during the decoding process, thereby enhancing the robustness of the model.

[0156] For example, the different parameter types, configurations, and corresponding effects of the decoder can be shown in Table 2 below:

[0157] Table 2

[0158]

[0159] Optionally, the loss function can be a combination of the cross-entropy loss function and the Dice loss function to optimize color accuracy and biometric fidelity.

[0160] pass To address the constraints of calculating basic color accuracy and skin tone realism, Dice Loss optimization can be performed using pre-defined model feature maps to enhance facial skin tone consistency. Cross-entropy loss is used to ensure the accuracy of overall pixel color distribution.

[0161] For example, different data augmentation types, designs, and their corresponding effects can be shown in Table 3 below:

[0162] Table 3

[0163]

[0164] For example, different training parameters and their corresponding configurations can be shown in Table 4 below:

[0165] Table 4

[0166]

[0167] Optionally, the output layer tanh activation function can be constrained to the range of RGB values ​​from -1 to 1 for color calibration. The processing system can also perform adaptive histogram equalization (CLAHE) and lip / pupil protection based on semantic segmentation (pre-trained UNet model).

[0168] The above are the method embodiments provided in this application. The apparatus provided in this application will be described below.

[0169] Figure 3 This is a schematic diagram of the structure of an image colorization processing device provided in this application, as shown below. Figure 3 As shown, the image colorization processing apparatus 300 provided in this embodiment includes: an extraction module 301, a reconstruction module 302, and an optimization module 303. Among them,

[0170] The extraction module 301 is used to extract features from a grayscale image through a neural network structure to generate a high-order semantic feature map; the neural network structure is a deep neural network composed of an encoder and a decoder.

[0171] Reconstruction module 302 is used to reconstruct the features of the high-order semantic feature map and generate three-channel image data;

[0172] The optimization module 303 is used to optimize the three-channel image data through a hybrid loss function to generate the target color image.

[0173] Optionally, the extraction module 301 is further configured to perform data augmentation processing on the grayscale image before performing feature extraction on the grayscale image through the neural network structure, and generate an enhanced grayscale image; the data augmentation processing specifically includes at least one of the following: Gaussian noise injection, motion blur simulation, and brightness jitter adjustment.

[0174] Optionally, the extraction module 301 is also used to adjust the brightness and saturation of the grayscale image through the HLS color space to generate a variety of enhanced images, wherein the HLS color space is a color representation method that includes brightness, hue, and saturation.

[0175] Optionally, the extraction module 301 is also used to extract features from the grayscale image through depthwise separable convolution to generate the high-order semantic feature map.

[0176] Optionally, the reconstruction module 302 is also used to perform channel concatenation of the feature maps of the corresponding layers in the encoder network and the feature maps of the decoder network through skip connections to generate a fused feature map; and to upsample the fused feature map through transposed convolution to generate three-channel image data.

[0177] Optionally, the reconstruction module 302 is further configured to upsample the fused feature map through transposed convolution, extract the feature mask map of the target region through a semantic segmentation network, and perform element-wise multiplication of the feature mask map with the three-channel image data to generate an enhanced target color image.

[0178] Optionally, the optimization module 303 is further configured to calculate the pixel distribution difference between the three-channel image data and the target color image using the cross-entropy loss function; calculate the local feature similarity between the three-channel image data and the target color image using the Dice loss function based on the output of the preset feature extraction network; and perform a weighted summation of the pixel distribution difference and the local feature similarity to generate the optimized target color image.

[0179] Optionally, the optimization module 303 is also used to optimize the three-channel image data by means of illumination invariance constraints to generate a target color image.

[0180] The image colorization processing apparatus provided in this embodiment can execute the methods provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0181] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 4 As shown, the electronic device 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 400 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0182] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0183] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0184] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0185] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0186] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0187] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0188] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0189] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0190] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0191] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0192] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0193] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0194] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0195] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0196] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0197] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image colorization processing method, characterized in that, include: Features are extracted from grayscale images using a neural network structure to generate high-order semantic feature maps; the neural network structure is a deep neural network composed of an encoder and a decoder. The higher-order semantic feature map is reconstructed to generate three-channel image data; The target color image is generated by optimizing the three-channel image data using a hybrid loss function.

2. The method according to claim 1, characterized in that, The step of reconstructing the high-order semantic feature map to generate three-channel image data includes: By using skip connections, the feature maps of the corresponding layers in the encoder network and the feature maps of the decoder network are concatenated through channels to generate a fused feature map. The fused feature map is upsampled by transposed convolution to generate three-channel image data.

3. The method according to claim 2, characterized in that, After upsampling the fused feature map through transposed convolution, the process includes: Feature mask images of target regions are extracted using a semantic segmentation network; The feature mask image is multiplied element-wise with the three-channel image data to generate an enhanced target color image.

4. The method according to claim 1, characterized in that, The step of optimizing the three-channel image data using a hybrid loss function to generate a target color image includes: The pixel distribution difference between the three-channel image data and the target color image is calculated using the cross-entropy loss function; The local feature similarity between the three-channel image data and the target color image is calculated based on the output of the preset feature extraction network using the Dice loss function. The pixel distribution differences and the local feature similarity are weighted and summed to generate the optimized target color image.

5. The method according to claim 1, characterized in that, Before performing feature extraction on the grayscale image using a neural network structure, the following steps are included: The grayscale image is subjected to data augmentation processing to generate an enhanced grayscale image; The data augmentation process specifically includes at least one of the following: Gaussian noise injection, motion blur simulation, and brightness jitter adjustment.

6. The method according to claim 5, characterized in that, The step of performing data augmentation processing on the grayscale image to generate an enhanced grayscale image includes: By adjusting the brightness and saturation of the grayscale image using the HLS color space, diverse enhanced images can be generated. The HLS color space is a color representation method that includes brightness, hue, and saturation.

7. The method according to claim 1, characterized in that, The feature extraction of grayscale images using a neural network structure includes: The grayscale image is used to extract features through depthwise separable convolution to generate the high-order semantic feature map.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The target color image is generated by optimizing the three-channel image data by constraining the illumination invariance.

9. An image colorization processing apparatus, characterized in that, include: The extraction module is used to extract features from grayscale images through a neural network structure to generate high-order semantic feature maps; the neural network structure is a deep neural network composed of an encoder and a decoder. The reconstruction module is used to reconstruct features from the high-order semantic feature map and generate three-channel image data. The optimization module is used to optimize the three-channel image data using a hybrid loss function to generate a target color image.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images