Low-illumination grayscale image colorization method

Abstract

The application discloses a low-illumination grayscale image colorization method, comprising the following steps: acquiring a to-be-processed image; identifying the scene type of the to-be-processed image by using a trained special scene recognition model; and performing colorization on the to-be-processed image by selecting a trained micro-light image colorization model suitable for the special scene according to the scene type. The low-illumination grayscale image colorization method significantly improves the grayscale image colorization effect of the special scene under low-illumination conditions, especially in the case of high-quality data set, the quality of the generated image is obviously improved; the CycleGAN is optimized, the color restoration effect is improved, the network structure is simplified, the calculation speed is improved, and an efficient and fast image colorization algorithm is provided for the special scene.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a method for colorizing low-light grayscale images. Background Technology

[0002] With the widespread application of night vision technology in military, security, and field exploration, achieving high-quality image perception and analysis under low-light conditions has become a core research challenge. Low-light colorization technology aims to improve image quality caused by insufficient illumination by enhancing the color information of images under low-light conditions. In dim light environments, the lack of light can lead to severe loss of image information, typically manifested as color loss and blurred details, making images appear dark and unrealistic. This phenomenon not only affects the visual experience but also poses a challenge to practical applications, especially in fields such as security surveillance, nighttime operations, and medical imaging.

[0003] Traditional low-light image colorization methods are mainly divided into three types: pseudo-color enhancement, false-color synthesis, and true-color restoration. Grayscale image color encoding technology generates color images through digital mapping; however, the colors it presents only serve an indicative function and differ from the characteristics of human vision, easily leading to eye fatigue. While multispectral fusion-based false-color technology integrates data from different bands and applies color transfer principles, exhibiting superior naturalness in low-light and infrared image fusion applications, its colors do not conform to human visual characteristics and lack image detail. These methods primarily rely on image enhancement or simple mapping techniques, which, while improving image quality to some extent, still face problems such as color distortion, loss of detail, and insufficient adaptability to environmental characteristics in low-light environments, especially in complex scenes. Furthermore, most existing night vision equipment primarily uses monochrome images, failing to accurately reproduce the spectral information of real scenes, which greatly limits its practical application value in complex combat environments.

[0004] Chinese invention patent CN107481183B, published on December 15, 2017, discloses a low-light image colorization method based on multi-dimensional data association rules, which mainly uses digital mapping to achieve low-light colorization effects. Chinese invention patent application CN119006655A, published on November 22, 2024, discloses a multi-level image colorization network system and method based on a lightweight attention structure, which uses fusion and color matching methods combined with a classification loss function to achieve more vibrant coloring effects. Neither of these methods achieves true color restoration. They lack specificity for special scenarios such as deserts and forests, resulting in poor application effects. Furthermore, the target samples in these colorization studies suffer from a wide range, weak targeting, and poor adaptability to special scenarios.

[0005] In summary, existing technologies suffer from poor color reproduction of grayscale images in special scenes under low lighting conditions. Summary of the Invention

[0006] The purpose of this invention is to provide a method for colorizing grayscale images in low light conditions, which solves the problem of poor color reproduction of grayscale images in special scenes under low light conditions in the prior art.

[0007] The technical solution adopted in this invention is a method for colorizing low-light grayscale images, comprising the following steps:

[0008] Step 1: Obtain the image to be processed;

[0009] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0010] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type and colorize the image to be processed.

[0011] The invention is further characterized by:

[0012] The trained special scene recognition model is obtained through the following steps:

[0013] Step A1: Collect low-light environment data for special scenarios;

[0014] Step A2: Perform data augmentation and preprocessing on the low-light environment data;

[0015] Step A3: Pre-train the special scene recognition model using a public dataset;

[0016] Step A4: Use the data augmentation and preprocessed low-light environment data to fine-tune and optimize the pre-trained special scene recognition model to obtain a well-trained special scene recognition model.

[0017] Low-light environment data is acquired through a data acquisition device, which includes a controller. The controller is connected to a low-light illuminance meter and an image acquisition module. The image acquisition module includes a monochrome CMOS camera.

[0018] Step A2 includes data augmentation and preprocessing, which includes data augmentation and image preprocessing. Data augmentation includes cropping, rotating, and enlarging the image. Image preprocessing includes normalization and denoising.

[0019] The special scene recognition model is based on CNN combined with transfer learning to achieve environmental recognition. The basic architecture of the special scene recognition model is the residual neural network ResNet18. When fine-tuning the special scene recognition model, cross-entropy is used as the loss function, and the Adam optimizer is used to update parameters. A dynamic learning rate adjustment strategy is implemented to optimize the training effect. When fine-tuning the special scene recognition model, the model performance is evaluated using a validation set after each round of training. Accuracy, precision, recall, and F1 score are calculated, and hyperparameters are tuned based on the evaluation results.

[0020] The formula for calculating the loss function of the special scene recognition model is as follows:

[0021]

[0022] in, Loss The loss function represents the loss function used when fine-tuning and optimizing a special scene recognition model. ，C Indicates the total number of categories. i Indicates the first i Categories y i Indicates the first i Samples of each category, P i This indicates that the model predicts the sample belongs to the first... i The probability of a class.

[0023] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0024] Step B1: Collect environmental data under special scenarios;

[0025] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0026] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0027] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0028] Environmental data includes low-light environmental data and visible light environmental data; low-light environmental data includes low-light data for forest environments and low-light data for desert environments; visible light environmental data includes visible light data for forest environments and visible light data for desert environments.

[0029] Environmental data is obtained through a data acquisition device, which includes a controller. The controller is connected to a low-light illuminance meter and an image acquisition module. The image acquisition module includes a monochrome CMOS camera, and a filter is set on the monochrome CMOS camera.

[0030] Data augmentation includes cropping, rotating, and enlarging images; preprocessing includes normalization and noise reduction.

[0031] Step B3 includes a low-light image colorization model for desert environments, which adds a perceptual loss function and a gradient penalty function to the traditional CycleGAN network; and a low-light image colorization model for forest environments, which adds a perceptual loss function, a gradient penalty function, and a color transfer module to the traditional CycleGAN network.

[0032] Model performance was evaluated using structural similarity, peak signal-to-noise ratio, and a natural image quality evaluator.

[0033] Low-light image colorization models include low-light image colorization models for forest scenes and low-light image colorization models for desert scenes;

[0034] The low-light image colorization model for forest scenes includes a deep generator, which contains a 28-layer network. The core of the deep generator adopts a ResNet structure with 9 residual blocks. It uses a three-stage architecture of preprocessing, downsampling and residual feature transformation, combined with a transposed convolutional decoder, and incorporates reflection filling technology to reduce artifacts. The final output is the processed result normalized by 7×7 convolution and Tanh function.

[0035] The standardized formula for calculating the Tanh function is shown below:

[0036]

[0037] in, This represents the output after the input data x has been standardized using the hyperbolic tangent function. Indicates input data;

[0038] The low-light image colorization model for forest scenes also includes a discriminator. The discriminator uses a PatchGAN architecture with a 70×70 receptive field for local evaluation and obtains the global discrimination result through matrix mean aggregation.

[0039] The low-light image colorization model for forest scenes also includes a perceptual consistency loss function; the formula for calculating the perceptual consistency loss function is shown below:

[0040]

[0041] ,

[0042] in, This represents the perceptual consistency loss function, which measures the similarity between the generated image and the original image in the perceptual feature space. This indicates that input data will be provided. From Domain Convert to domain , This indicates that input data will be provided. From Domain Convert to domain , Indicates from domain Input data obtained by mid-sampling , Indicates from domain Input data obtained by mid-sampling , ϕ This represents the feature extraction function of the pre-trained network; Denotes the L1 norm and represents the perceptual difference between features. Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features, Indicates input data Perceptual characteristics; Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features; Indicates input data Perceptual characteristics;

[0043] The low-light image colorization model for forest scenes also includes a discriminator, and the formula for calculating the gradient constraint of the discriminator is shown below:

[0044] ,

[0045] in, This represents the gradient penalty loss, used to stabilize the training of generative adversarial networks (GANs) and prevent gradient vanishing or exploding. This represents the discriminator, which determines whether the input image comes from a real image in the dataset or from an image generated by the generator. Indicates distribution from the middle Medium-sampled images ; Represents a real image. Represents the linear interpolation between the real image and the generated image. This represents the gradient of the discriminator on the interpolated image. This represents the L2 norm.

[0046] Low-light image colorization models include low-light image colorization models for forest scenes and low-light image colorization models for desert scenes;

[0047] The low-light image colorization model for desert scenes includes a depth generator, which contains a 28-layer network. The core of the depth generator adopts a ResNet structure with 9 residual blocks. It uses a three-stage architecture of preprocessing, downsampling and residual feature transformation, combined with a transposed convolutional decoder, and incorporates reflection filling technology to reduce artifacts. The final output is the processed result normalized by 7×7 convolution and Tanh function.

[0048] The standardized formula for calculating the Tanh function is shown below:

[0049]

[0050] in, This represents the output after the input data x has been standardized using the hyperbolic tangent function. Indicates input data;

[0051] The low-light image colorization model for desert scenes also includes a discriminator. The discriminator uses a PatchGAN architecture with a 70×70 receptive field for local evaluation and obtains the global discrimination result through matrix mean aggregation.

[0052] The low-light image colorization model for desert scenes also includes a perceptual consistency loss function; the formula for calculating the perceptual consistency loss function is shown below:

[0053]

[0054] ,

[0055] in, This represents the perceptual consistency loss function, which measures the similarity between the generated image and the original image in the perceptual feature space. This indicates that input data will be provided. From Domain Convert to domain , This indicates that input data will be provided. From Domain Convert to domain , Indicates from domain Input data obtained by mid-sampling , Indicates from domain Input data obtained by mid-sampling , ϕ This represents the feature extraction function of the pre-trained network; Denotes the L1 norm and represents the perceptual difference between features. Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features, Indicates input data Perceptual characteristics; Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features; Indicates input data Perceptual characteristics;

[0056] The low-light image colorization model for desert scenes also includes a discriminator, and the formula for calculating the gradient constraint of the discriminator is shown below:

[0057] ,

[0058] in, This represents the gradient penalty loss, used to stabilize the training of generative adversarial networks (GANs) and prevent gradient vanishing or gradient exploding. This represents the discriminator, which determines whether the input image comes from a real image in the dataset or from an image generated by the generator. Indicates distribution from the middle Medium-sampled images ; Represents a real image. Represents the linear interpolation between the real image and the generated image. This represents the gradient of the discriminator on the interpolated image. Represents the L2 norm;

[0059] The low-light image colorization model for desert scenes also includes a color transfer module. The color transfer module uses Reinhard color transfer technology to achieve color restoration through LAB space transformation and statistical feature matching, while combining cycle consistency loss and adversarial loss to ensure the quality of transformation.

[0060] The formula for Reinhard color transfer technology is expressed as follows:

[0061] ,

[0062] in, These represent the LAB color space values ​​of the output image after color transfer. Quantity; These represent the target image, i.e., the image to be processed. Quantity; These represent the target image in Mean across dimensions; These represent the target image in Standard deviation in dimensions; This indicates that the source image, i.e., the reference image, is in Mean across dimensions; Indicates the source image in Standard deviation in dimensions.

[0063] The beneficial effects of this invention are as follows: This invention significantly improves the colorization effect of grayscale images in special scenes under low illumination conditions, especially when the dataset quality is high, the quality of the generated images is significantly improved; This invention optimizes CycleGAN (Cycle Generative Adversarial Network), which simplifies the network structure and improves the calculation speed while improving the color restoration effect, providing an efficient and fast image colorization algorithm for special scenes; During the data acquisition process, a black and white camera with higher resolution is used, which not only improves the image clarity, but also provides richer raw data support for subsequent color restoration; Because RGB filters are used during the data acquisition process, and near-infrared images are converted into R, G, and B images respectively and then synthesized during deep learning, the subsequent image colorization process presents the true color of the target object. Attached Figure Description

[0064] Figure 1 This is a flowchart of the low-light grayscale image colorization method of the present invention;

[0065] Figure 2 This is a schematic diagram of the image acquisition device in this invention.

[0066] In the diagram, 1. Filter; 2. Low-light illuminance meter; 3. Black and white CMOS camera; 4. Controller. Detailed Implementation

[0067] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0068] Example 1

[0069] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0070] Step 1: Obtain the image to be processed;

[0071] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0072] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type and colorize the image to be processed.

[0073] Example 2

[0074] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0075] Step 1: Obtain the image to be processed;

[0076] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0077] The trained special scene recognition model is obtained through the following steps:

[0078] Step A1: Collect low-light environment data for special scenarios;

[0079] Step A2: Perform data augmentation and preprocessing on the low-light environment data;

[0080] Step A3: Pre-train the special scene recognition model using a public dataset;

[0081] Step A4: Use the data augmentation and preprocessed low-light environment data to fine-tune and optimize the pre-trained special scene recognition model to obtain a well-trained special scene recognition model.

[0082] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type, and colorize the image to be processed.

[0083] Example 3

[0084] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0085] Step 1: Obtain the image to be processed;

[0086] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0087] The trained special scene recognition model is obtained through the following steps:

[0088] Step A1: Collect low-light environment data for special scenarios;

[0089] Low-light environment data is acquired through a data acquisition device, which includes a controller 4. The controller 4 is connected to a low-light illuminance meter 2 and an image acquisition module, respectively. The image acquisition module includes a monochrome CMOS camera 3.

[0090] Step A2: Perform data augmentation and preprocessing on the low-light environment data;

[0091] Data augmentation and preprocessing include data augmentation and image preprocessing; data augmentation includes cropping, rotating, and enlarging images; image preprocessing includes normalization and noise reduction.

[0092] Step A3: Pre-train the special scene recognition model using a public dataset;

[0093] Step A4: Use the data augmentation and preprocessed low-light environment data to fine-tune and optimize the pre-trained special scene recognition model to obtain a well-trained special scene recognition model.

[0094] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type and colorize the image to be processed.

[0095] Example 4

[0096] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0097] Step 1: Obtain the image to be processed;

[0098] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0099] The trained special scene recognition model is obtained through the following steps:

[0100] Step A1: Collect low-light environment data for special scenarios;

[0101] Step A2: Perform data augmentation and preprocessing on the low-light environment data;

[0102] Step A3: Pre-train the special scene recognition model using a public dataset;

[0103] Step A4: Use the data augmentation and preprocessed low-light environment data to fine-tune and optimize the pre-trained special scene recognition model to obtain a well-trained special scene recognition model.

[0104] The special scene recognition model is based on CNN combined with transfer learning to achieve environmental recognition. The basic architecture of the special scene recognition model is the residual neural network ResNet18. When fine-tuning the special scene recognition model, cross-entropy is used as the loss function, and the Adam optimizer is used to update parameters. A dynamic learning rate adjustment strategy is implemented to optimize the training effect. When fine-tuning the special scene recognition model, the model performance is evaluated using a validation set after each round of training. Accuracy, precision, recall, and F1 score are calculated, and hyperparameters are tuned based on the evaluation results.

[0105] The formula for calculating the loss function of the special scene recognition model is as follows:

[0106]

[0107] in, Loss The loss function represents the loss function used when fine-tuning and optimizing a special scene recognition model. ，C Indicates the total number of categories. i Indicates the first i Categories y i Indicates the first i Samples of each category, P i This indicates that the model predicts the sample belongs to the first... i The probability of a class.

[0108] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type, and colorize the image to be processed.

[0109] Example 5

[0110] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0111] Step 1: Obtain the image to be processed;

[0112] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0113] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type, and colorize the image to be processed.

[0114] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0115] Step B1: Collect environmental data under special scenarios;

[0116] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0117] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0118] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0119] Example 6

[0120] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0121] Step 1: Obtain the image to be processed;

[0122] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0123] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type, and colorize the image to be processed.

[0124] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0125] Step B1: Collect environmental data under special scenarios;

[0126] Environmental data includes low-light environmental data and visible light environmental data; low-light environmental data includes low-light data for forest environments and low-light data for desert environments; visible light environmental data includes visible light data for forest environments and visible light data for desert environments.

[0127] Environmental data through, for example Figure 2 The data acquisition device shown includes a controller 4, which is connected to a low-light illuminance meter 2 and an image acquisition module. The image acquisition module includes a black and white CMOS camera 3, and a filter 1 is provided on the black and white CMOS camera 3.

[0128] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0129] Data augmentation includes cropping, rotating, and enlarging images; preprocessing includes normalization and noise reduction.

[0130] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0131] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0132] Example 7

[0133] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0134] Step 1: Obtain the image to be processed;

[0135] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0136] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type and colorize the image to be processed.

[0137] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0138] Step B1: Collect environmental data under special scenarios;

[0139] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0140] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0141] For the low-light image colorization model in desert environments, a perceptual loss function and a gradient penalty function are added to the traditional CycleGAN network; for the low-light image colorization model in forest environments, a perceptual loss function, a gradient penalty function, and a color transfer module are added to the traditional CycleGAN network.

[0142] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0143] Model performance was evaluated using structural similarity, peak signal-to-noise ratio, and a natural image quality evaluator.

[0144] Example 8

[0145] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0146] Step 1: Obtain the image to be processed;

[0147] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0148] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type, and colorize the image to be processed.

[0149] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0150] Step B1: Collect environmental data under special scenarios;

[0151] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0152] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0153] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0154] Low-light image colorization models include low-light image colorization models for forest scenes and low-light image colorization models for desert scenes;

[0155] The low-light image colorization model for forest scenes includes a deep generator, which contains a 28-layer network. The core of the deep generator adopts a ResNet structure with 9 residual blocks. It uses a three-stage architecture of preprocessing, downsampling and residual feature transformation, combined with a transposed convolutional decoder, and incorporates reflection filling technology to reduce artifacts. The final output is the processed result normalized by 7×7 convolution and Tanh function.

[0156] The standardized formula for calculating the Tanh function is shown below:

[0157]

[0158] in, This represents the output after the input data x has been standardized using the hyperbolic tangent function. Indicates input data;

[0159] The low-light image colorization model for forest scenes also includes a discriminator. The discriminator uses a PatchGAN architecture with a 70×70 receptive field for local evaluation and obtains the global discrimination result through matrix mean aggregation.

[0160] The low-light image colorization model for forest scenes also includes a perceptual consistency loss function; the formula for calculating the perceptual consistency loss function is shown below:

[0161]

[0162] ,

[0163] in, This represents the perceptual consistency loss function, which measures the similarity between the generated image and the original image in the perceptual feature space. This indicates that input data will be provided. From Domain Convert to domain , This indicates that input data will be provided. From Domain Convert to domain , Indicates from domain Input data obtained by mid-sampling , Indicates from domain Input data obtained by mid-sampling , ϕ This represents the feature extraction function of the pre-trained network; Denotes the L1 norm and represents the perceptual difference between features. Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features, Indicates input data Perceptual characteristics; Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features; Indicates input data Perceptual characteristics;

[0164] The low-light image colorization model for forest scenes also includes a discriminator, and the formula for calculating the gradient constraint of the discriminator is shown below:

[0165] ,

[0166] in, This represents Gradient Penalty Loss, used to stabilize the training of Generative Adversarial Networks (GANs) and prevent gradient vanishing or gradient exploding. This represents the discriminator, which determines whether the input image comes from a real image in the dataset or from an image generated by the generator. Indicates distribution from the middle Medium-sampled images ; Represents a real image. Represents the linear interpolation between the real image and the generated image. This represents the gradient of the discriminator on the interpolated image. This represents the L2 norm.

[0167] Example 9

[0168] This embodiment proposes a method for colorizing low-light grayscale images, including the following steps:

[0169] Step 1: Obtain the image to be processed;

[0170] Step 2: Use the trained special scene recognition model to identify the scene type of the image to be processed;

[0171] Step 3: Select a suitable, well-trained low-light image colorization model for the specific scene based on the scene type and colorize the image to be processed.

[0172] The trained low-light image colorization model for specific scenarios is obtained through the following steps:

[0173] Step B1: Collect environmental data under special scenarios;

[0174] Step B2: Perform data augmentation and preprocessing on environmental data in special scenarios;

[0175] Step B3: Improve CycleGAN to obtain a low-light image colorization model for specific scenarios;

[0176] Step B4: Input the environmental data of the special scene into the low-light image colorization model for the special scene for optimization training, and then judge the model performance. When the judgment result meets the set conditions, the trained low-light image colorization model for the special scene is obtained.

[0177] Low-light image colorization models include low-light image colorization models for forest scenes and low-light image colorization models for desert scenes;

[0178] The low-light image colorization model for forest scenes includes a deep generator, which contains a 28-layer network. The core of the deep generator adopts a ResNet structure with 9 residual blocks. It uses a three-stage architecture of preprocessing, downsampling and residual feature transformation, combined with a transposed convolutional decoder, and incorporates reflection filling technology to reduce artifacts. The final output is the processed result normalized by 7×7 convolution and Tanh function.

[0179] The standardized formula for calculating the Tanh function is shown below:

[0180]

[0181] in, This represents the output after the input data x has been standardized using the hyperbolic tangent function. Indicates input data;

[0182] The low-light image colorization model for forest scenes also includes a discriminator. The discriminator uses a PatchGAN architecture with a 70×70 receptive field for local evaluation and obtains the global discrimination result through matrix mean aggregation.

[0183] The low-light image colorization model for forest scenes also includes a perceptual consistency loss function; the formula for calculating the perceptual consistency loss function is shown below:

[0184]

[0185] ,

[0186] in, This represents the perceptual consistency loss function, which measures the similarity between the generated image and the original image in the perceptual feature space. This indicates that input data will be provided. From Domain Convert to domain , This indicates that input data will be provided. From Domain Convert to domain , Indicates from domain Input data obtained by mid-sampling , Indicates from domain Input data obtained by mid-sampling , ϕ This represents the feature extraction function of the pre-trained network; Denotes the L1 norm and represents the perceptual difference between features. Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features, Indicates input data Perceptual characteristics; Indicates passing first Will From Domain Convert to domain Then through Remove it from the domain Convert back to domain Finally passed Extracting perceptual features; Indicates input data Perceptual characteristics;

[0187] The low-light image colorization model for forest scenes also includes a discriminator, and the formula for calculating the gradient constraint of the discriminator is shown below:

[0188] ,

[0189] in, This represents Gradient Penalty Loss, used to stabilize the training of Generative Adversarial Networks (GANs) and prevent gradient vanishing or gradient exploding. This represents the discriminator, which determines whether the input image comes from a real image in the dataset or from an image generated by the generator. Indicates distribution from the middle Medium-sampled images ; Represents a real image. Represents the linear interpolation between the real image and the generated image. This represents the gradient of the discriminator on the interpolated image. This represents the L2 norm.

[0190] This invention innovatively utilizes a deep learning algorithm and a CycleGAN network to achieve low-light colorization in specific environments. It constructs a mapping relationship between 700-800nm ​​near-infrared images and visible light R, G, B images, achieving the effect of restoring true visible light colors using only 700-800nm ​​near-infrared grayscale images. Compared to traditional methods, this represents a dual breakthrough in scene adaptability and color reproduction accuracy. Furthermore, to better restore the true color information of images in special low-light environments, this invention optimizes the generator architecture based on the traditional CycleGAN, increasing the number of residual blocks and incorporating a perceptual loss function and gradient penalty to improve the visual quality of the colorized images and the stability of the network. For desert environments, a color transfer strategy is added to further optimize the colorization effect, addressing the shortcomings of traditional colorization methods under low-light conditions. Simultaneously, in environment recognition, transfer learning is performed using a pre-trained ResNet-18 network, simplifying the network structure and improving the speed of environment recognition. Furthermore, using a monochrome camera with a higher resolution than a traditional color camera during image acquisition not only improves image clarity but also provides richer raw data support for subsequent color restoration.

[0191] In this embodiment of the invention, low-light values ​​are used to calibrate the environment. A low-light illuminance meter 2 is used to measure the ambient illuminance. A filter 1 is added in front of a monochrome CMOS camera 3 to acquire images. The image data is then transmitted to a controller 4 for subsequent algorithm processing. The dataset for image colorization focuses on solving the problem of converting near-infrared images to visible light images under low-light conditions. Under the same low-light conditions (10... -1 ~10 -5 Near-infrared images are acquired using filter 1 (specifically, 700–800 nm, 800–900 nm, 900–1000 nm, etc.) as an environmental recognition dataset. Simultaneously, to provide a realistic reference for colorization, R, G, and B channel images are acquired at the same scene location during the daytime using a camera and multi-band filters (specifically, 400–500 nm, 500–600 nm, 600–700 nm) as a low-light colorization dataset. Each dataset contains four images: one near-infrared image and three corresponding visible light sub-channel images. Multiple datasets are acquired for each scene (specifically, desert / forest, etc.), totaling 2000–6000 datasets.

[0192] The data collected in this invention aims to support environmental recognition and low-light image colorization research, covering typical natural environments such as deserts and forests, and is dedicated to the research and implementation of true-color restoration technology for low-light grayscale images. The special scene recognition model in this invention is based on CNN combined with transfer learning to achieve environmental recognition. An 18-layer residual neural network, ResNet18, is selected as the basic architecture, and pre-trained using public datasets such as ImageNet1000 to build a basic model. Subsequently, the pre-trained model is transferred to special combat scenarios, and forest and desert-related datasets are introduced to fine-tune and optimize the model, thereby improving its recognition performance in these specific scenarios. During training, the special scene recognition model uses cross-entropy as the loss function, combined with the Adam optimizer for parameter updates, and implements a dynamic learning rate adjustment strategy to optimize the training effect. The entire training process continuously sets the number of training epochs. After each epoch, the model performance is evaluated using a validation set, calculating key indicators such as accuracy, precision, and recall. Based on the evaluation results, hyperparameter tuning is performed to ensure optimal model performance in specific environments.

[0193] Accuracy ( Accuracy As one of the most widely used performance evaluation metrics, accuracy directly reflects the classification model's ability to distinguish between different categories. Generally, a higher accuracy value means that the model has better overall recognition performance across different categories. The definition is as follows:

[0194] ,

[0195] in, TP This refers to the situation where the model accurately predicts a positive example as a positive example. TN This refers to the situation where the model accurately predicts a negative example as a negative example. FP This refers to the situation where the model incorrectly predicts a negative class as a positive class. FN This refers to a situation where the model incorrectly predicts a positive class as a negative class.

[0196] Accuracy ( Precision The positive metric reflects the proportion of true positive examples among the samples classified as positive by the model. The higher the value, the more reliable the model's positive prediction results are. It is defined as follows:

[0197] ,

[0198] in, TP This refers to the situation where the model accurately predicts a positive example as a positive example. FP This refers to a situation where the model incorrectly predicts a negative class as a positive class.

[0199] Recall rate ( RecallThe metric is used to measure the model's ability to correctly identify positive class samples. The higher the value of this metric, the stronger the model's ability to detect the target class. Its definition is as follows:

[0200] ,

[0201] in, TP This refers to the situation where the model accurately predicts a positive example as a positive example. FN This refers to a situation where the model incorrectly predicts a positive class as a negative class.

[0202] F1 score ( F1Score As a weighted harmonic average of precision and recall, a higher value of this metric indicates a better balance between accurately identifying positive samples (high precision) and covering as many positive samples as possible (high recall). Its calculation formula is:

[0203] ,

[0204] in, Recall Indicates recall rate, Precision Indicates precision.

[0205] This invention presents a low-light image colorization model based on an optimized CycleGAN network. This model enables colorization of low-light images in various environments. The training effect of the model is evaluated through discussion of the loss curve, the training process is optimized, and hyperparameters are adjusted. When the discrimination result meets a fixed threshold range, a colorized image is output. In low-light image colorization, this invention employs an optimized CycleGAN generative adversarial network to learn the mapping relationship between near-infrared and visible light. To further improve performance, the low-light image colorization model introduces perceptual consistency loss and discriminator gradient constraints. For desert environments with weak near-infrared response, Reinhard color transfer technology is integrated into the forest scene method, achieving color restoration through LAB spatial transformation and statistical feature matching. Simultaneously, cyclic consistency loss and adversarial loss are combined to ensure conversion quality. During training, the hyperparameters are optimized by analyzing the loss curve, achieving continuous improvement in model performance.

[0206] The low-light image colorization model in this invention adopts structural similarity ( SSIM Peak signal-to-noise ratio (PSNR) PSNR ) and natural image quality evaluator ( NIQE This is used to evaluate the performance of low-light image colorization models. Structural similarity considers brightness, contrast, and structural information; it is used to determine the similarity between two images, with higher similarity being better. SSIM The value indicates that two images are more similar in perceived quality, and is defined as follows:

[0207]

[0208] In the formula, and Average pixel value; and For variance; Covariance; and It is a constant.

[0209] Peak signal-to-noise ratio (PSNR) is used to measure the difference between the reconstructed image and the original image. It is commonly used to evaluate image compression and reconstruction quality, and its definition is as follows:

[0210]

[0211] ,

[0212] In the formula, MSE ( x , y ) represents the mean square error. x , y These represent the synthesized image output by the generator and the actual reference image of the corresponding target domain, respectively. and Pixels in the input and reference images respectively grayscale information, M Image length, N Indicates the width of the image.

[0213] Natural Image Quality Evaluator NIQE As a no-reference image quality evaluation metric, it requires no original reference image, has reasonable visual perception, and is defined as follows:

[0214] ,

[0215] In the formula, , The mean vector of the MVG model for the actual image and the distorted image; , These are the corresponding covariance matrices.

[0216] The environmental recognition dataset in this embodiment of the invention not only includes self-collected real-world scene data, but also incorporates high-quality publicly available online datasets as a supplement. In low-light (10... -1 ~10 -3Under lux conditions, the average accuracy of forest environment identification reached 93.8%. The special scene identification model performed excellently in all performance indicators, with an average accuracy of approximately 92.41%, precision of approximately 93.22%, F1 score of approximately 91.23%, and recall of approximately 90.84%. These results indicate that the special scene identification model not only outperforms traditional methods in terms of accuracy and precision but also has a significant advantage in training efficiency.

[0217] Taking a forest environment as an example, compared with other algorithms, the special scene recognition model proposed in this invention shows a systematic improvement in the low-light image colorization results. The relevant evaluation parameters for different algorithms in different forest environments are shown in Tables 1, 2, 3, and 4 below. Compared with SimGAN (Similarity Generative Adversarial Network), WGAN (Wasserstein GAN), and the original CycleGAN, the low-light image colorization model in this invention achieves optimization in three core indicators: SSIM structural similarity index, peak signal-to-noise ratio (PSNR), and Natural Image Quality Evaluator (NIQE).

[0218] Table 1 Comparison of different algorithms for forest environment scenario (a)

[0219]

[0220] Table 2 Comparison Results of Different Algorithms in Forest Environment Scenario (b)

[0221]

[0222] Table 3 Comparison Results of Different Algorithms in Forest Environment Scenario (c)

[0223]

[0224] Table 4 Comparison of different algorithms for forest environment scenario (d)

[0225]

[0226] In forest scenarios, this invention achieves a significant breakthrough in PSNR. Taking scenario (c) as an example, its PSNR value reaches 23.445dB, which is 17% higher than the original CycleGAN. The average PSNR across the four scenarios reaches 21.781dB, which is 8.2% higher than the benchmark algorithm on average, and the NIQE value remains stable in the range of 0.002-0.017, comparable to SimGAN.

[0227] Taking desert environments as an example, the relevant evaluation parameters for different algorithms in different scenarios are shown in Tables 5, 6, 7, and 8 below. Compared to the baseline algorithm, the NIQE value of this invention is systematically reduced: in scenario (a), it reaches 0.0289, a reduction of 11.1% compared to the original CycleGAN. In extreme lighting scenario (c), the NIQE value drops to 0.011, achieving a 52.2% optimization compared to WGAN, verifying the improved model's effect on enhancing the naturalness of desert images. The average PSNR for the four scenarios reaches 26.467 dB, an improvement of 53.8% compared to traditional methods, and the SSIM remains stable in the range of 0.891-0.934, indicating that it can effectively maintain the dune texture structure and color continuity even under strong light interference.

[0228] Table 5 Comparison of different algorithms in desert environment scenario (a)

[0229]

[0230] Table 6 Comparison of different algorithms in desert environment scenario (b)

[0231]

[0232] Table 7 Comparison Results of Different Algorithms in Desert Environment Scene (c)

[0233]

[0234] Table 8 Comparison Results of Different Algorithms for Desert Environment Scene (d)

[0235]

[0236] The special scene recognition model in this invention demonstrates significant advantages in adaptability to diverse environments. For forest environments, the enhancement scheme based on the improved CycleGAN effectively solves the structural distortion problem in areas where branches and leaves intersect, improving the ability to preserve details in dark areas by approximately 40%, by changing the number of residual blocks and the size of the convolutional kernel. In desert scenes, the newly added color transfer module successfully overcomes the color shift caused by highly reflective sand particles, improving color restoration accuracy by more than 25%, outperforming mainstream algorithms.

[0237] This invention addresses special scenarios such as forests and deserts by optimizing the CycleGAN generative adversarial network and learning the mapping relationship between near-infrared and visible light, thereby achieving colorization of low-light images. While generative adversarial networks (GANs) such as PatchGAN, CycleGAN, and basic GANs can also be used for image colorization, this invention simplifies the network structure based on the characteristics of forests and deserts, achieving higher peak signal-to-noise ratio and structural similarity, further demonstrating its superior colorization results. Furthermore, in specific implementations, a 700-800 nm filter is used to acquire near-infrared images during data acquisition. However, filters in any near-infrared band, such as 800-900 nm or 900-1000 nm, can also be used for near-infrared image acquisition.

Claims

1. A method of colorizing low-light grayscale images, characterized by, The method comprises the following steps: Step one, obtaining an image to be processed; Step two, identifying the scene type of the image to be processed using a trained special scene recognition model; Step three, selecting a trained micro-light image colorization model suitable for the special scene according to the scene type to colorize the image to be processed; The trained micro-light image colorization model for special scenes is obtained by training an improved CycleGAN model based on environmental data in special scenes; wherein for desert environments, a perception loss function, a gradient penalty function and a color transfer module are added to the traditional CycleGAN network; The environmental data includes micro-light environmental data and visible light environmental data; the micro-light environmental data includes near-infrared images, and the visible light environmental data includes three sub-channel images corresponding to each near-infrared image.

2. The low-illumination grayscale image colorization method of claim 1, wherein, The trained special scene recognition model is obtained by the following steps: Step A1, collecting micro-light environmental data in special scenes; Step A2, data augmentation and preprocessing of the micro-light environmental data; Step A3, pre-training the special scene recognition model using a public data set; Step A4, fine-tuning and optimizing the pre-trained special scene recognition model using the data augmented and preprocessed micro-light environmental data to obtain the trained special scene recognition model.

3. The low-illumination grayscale image colorization method of claim 2, wherein, The micro-light environmental data is collected by a data collection device, and the data collection device comprises a controller (4), wherein the controller (4) is connected with a micro-light illuminometer (2) and an image collection module, and the image collection module comprises a black and white CMOS camera (3); The data augmentation and preprocessing in step A2 includes data augmentation processing and image preprocessing; the data augmentation processing includes image cropping, rotating and enlarging; and the image preprocessing includes normalization processing and denoising processing.

4. The low-illumination grayscale image colorization method of claim 2, wherein, The special scene recognition model realizes environment recognition based on CNN combined with transfer learning, and the basic framework of the special scene recognition model is a residual neural network ResNet18; when the special scene recognition model is fine-tuned and optimized, cross-entropy is used as a loss function, the Adam optimizer is used for parameter updating, and a dynamic learning rate adjustment strategy is implemented to optimize the training effect; when the special scene recognition model is fine-tuned and optimized, the model performance is evaluated using a validation set after each round of training, the accuracy, precision, recall and F1 score are calculated, and the hyperparameters are optimized based on the evaluation results.

5. The low-illumination grayscale image colorization method of claim 4, wherein, The calculation formula of the loss function of the special scene recognition model is as follows: wherein, Loss represents a loss function when the special scene recognition model is fine-tuned and optimized ，C represents the total number of categories, i represents the i th category, y i represents the i th category sample, P i represents the probability that the model predicts that the sample belongs to the i th category.

6. The low-illumination grayscale image colorization method of claim 1, wherein, The trained micro-light image colorization model for special scenes is obtained by the following steps: Step B1, collecting environmental data in special scenes; Step B2, data augmentation and preprocessing of the environmental data in special scenes; Step B3, improving CycleGAN combined with special scenes to obtain a micro-light image colorization model for special scenes; Step B4, the environmental data in the special scene is input into the micro-light image colorization model for the special scene for optimization training, and the model performance is judged. When the judgment result meets the set condition, the trained micro-light image colorization model for the special scene is obtained.

7. The low-illumination grayscale image colorization method of claim 6, wherein, The micro-light environment data includes forest environment micro-light data and desert environment micro-light data; the visible light environment data includes forest environment visible light data and desert environment visible light data; The environmental data is obtained by a data acquisition device, and the data acquisition device includes a controller (4), the controller (4) is connected with a micro-light illuminometer (2) and an image acquisition module respectively, the image acquisition module includes a black and white CMOS camera (3), and a filter (1) is arranged on the black and white CMOS camera (3) in cooperation; The data enhancement includes image cropping, rotation and enlargement processing; the preprocessing includes normalization processing and denoising processing.

8. The low-illumination grayscale image colorization method of claim 6, wherein, Step B3 includes a micro-light image colorization model for the forest environment, and a perception loss function and a gradient penalty function are added to the traditional CycleGAN network; The model performance is evaluated by structural similarity, peak signal-to-noise ratio and natural image quality evaluator.

9. The low-illumination grayscale image colorization method of claim 6, wherein, The micro-light image colorization model includes a micro-light image colorization model for the forest scene and a micro-light image colorization model for the desert scene; The micro-light image colorization model for the forest scene includes a depth generator, the depth generator includes a 28-layer network, the depth generator core adopts a ResNet structure with 9 residual blocks, and a transposed convolution decoder is cooperated through a three-stage architecture of preprocessing, down-sampling and residual feature conversion, combined with a reflection padding technology to reduce artifacts, and finally outputs a processing result standardized by 7×7 convolution and Tanh function; The calculation formula of the Tanh function standardization is as follows: wherein, represents the output of the input data x normalized by the hyperbolic tangent function, represents the input data; The micro-light image colorization model for the forest scene further includes a discriminator, the discriminator adopts a PatchGAN architecture with a 70×70 receptive field for local evaluation, and obtains a global discrimination result through matrix mean aggregation; The micro-light image colorization model for the forest scene further includes a perception consistency loss function, and the calculation formula of the perception consistency loss function is as follows: ， wherein, represents a perceptual consistency loss function measuring the similarity of the generated image and the original image in the perceptual feature space, represents converting input data from domain to domain , represents converting input data from domain to domain , represents input data sampled from domain , represents input data sampled from domain , represents a feature extraction function of a pre-trained network; represents L1 norm and represents the perceptual difference between features, represents first converting from domain to domain , then converting it from domain back to domain , and finally extracting perceptual features by represents the perceptual features of input data ; represents first converting from domain to domain , then converting it from domain back to domain , and finally extracting perceptual features by represents the perceptual features of input data ;​​​​​​ The discriminator of the micro-light image colorization model for the forest scene further includes a gradient constraint, and the calculation formula of the gradient constraint is as follows: ， wherein, represents a gradient penalty loss for stabilizing the training of a generative adversarial network (GAN) to prevent gradient vanishing or gradient explosion; represents a discriminator that judges whether an input image is from a real image of a dataset or from a generated image of a generator; represents an image sampled from a middle distribution ; represents a real image, represents a linear interpolation of a real image and a generated image, represents a gradient of the discriminator on the interpolated image, represents an L2 norm.​ 10. The low-illumination grayscale image colorization method of claim 6, wherein, The micro-light image colorization model includes a micro-light image colorization model for the forest scene and a micro-light image colorization model for the desert scene; The micro-light image colorization model for the desert scene includes a depth generator, the depth generator includes a 28-layer network, the depth generator core adopts a ResNet structure with 9 residual blocks, and a transposed convolution decoder is cooperated through a three-stage architecture of preprocessing, down-sampling and residual feature conversion, combined with a reflection padding technology to reduce artifacts, and finally outputs a processing result standardized by 7×7 convolution and Tanh function; The calculation formula of the Tanh function standardization is as follows: wherein, represents the output of the input data x normalized by the hyperbolic tangent function, represents the input data; The low-light image colorization model for the desert scene further comprises a discriminator, which adopts a PatchGAN architecture with a 70*70 receptive field for local evaluation, and obtains a global discrimination result through matrix mean aggregation; The low-light image colorization model for the desert scene further comprises a perceptual consistency loss function, and a calculation formula of the perceptual consistency loss function is as follows: ， wherein, represents a perceptual consistency loss function measuring the similarity of the generated image and the original image in the perceptual feature space, represents converting input data from domain to domain , represents converting input data from domain to domain , represents input data sampled from domain , represents input data sampled from domain , represents a feature extraction function of the pre-trained network; represents L1 norm and represents the perceptual difference between features, represents first converting from domain to domain , then converting it from domain back to domain , and finally extracting perceptual features by represents perceptual features of input data ; represents first converting from domain to domain , then converting it from domain back to domain , and finally extracting perceptual features by represents perceptual features of input data ;​​​​​​ The low-light image colorization model for the desert scene further comprises a discriminator, and a calculation formula of gradient constraint of the discriminator is as follows: ， wherein, represents a gradient penalty loss for stabilizing the training of a generative adversarial network (GAN) to prevent gradient vanishing or gradient explosion; represents a discriminator that judges whether an input image is from a real image of a dataset or from a generated image of a generator; represents an image sampled from an intermediate distribution ; ; represents a real image, represents a linear interpolation of a real image and a generated image, represents a gradient of the discriminator on the interpolated image, represents an L2 norm; The low-light image colorization model for the desert scene further comprises a color transfer module, and a Reinhard color transfer technology is used in the color transfer module to realize color restoration through LAB space conversion and statistical feature matching, and meanwhile, a cycle consistency loss and an adversarial loss are combined to ensure conversion quality. A formula expression of the Reinhard color transfer technology is as follows: ， wherein, respectively represent the mean value of the target image in the dimension ; respectively represent the mean value of the target image in the dimension ; respectively represent the mean value of the target image in the dimension ; respectively represent the standard deviation of the target image in the dimension ; represent the mean value of the source image in the dimension ; represent the standard deviation of the source image in the dimension .

Images