Color image gray level conversion method, system and equipment based on quantitative perception and medium

Abstract

The invention discloses a color image gray level conversion method, system and device based on quantitative perception and a medium, and belongs to the field of gray level image processing. According to the color image gray level conversion method based on quantitative perception, feature learning is carried out based on the color image and the DCT de-noised gray level reference image by using a deep learning algorithm, the preliminary gray level optimization image is generated, and the accuracy of the brightness information and the pixel relation is ensured. On the basis, a quantitative perceptual loss (QPL) algorithm is further applied to map the color image and the gray level optimized image to a high-dimensional potential space, and high-level features of the images are captured. Through the method, the target grayscale image with better detail retention capability and perception quality can be generated, so that the target grayscale image is more natural visually and rich in details.

Description

Technical Field

[0001] The invention relates to a color image grayscale conversion method, system, device and medium based on quantitative perception, and belongs to the technical field of grayscale image processing. Background Art

[0002] In the field of image processing, color images and grayscale images have their own advantages and disadvantages in application. Color images can capture rich color information, but in certain specific scenarios (such as industrial inspection, medical imaging, etc.), grayscale images have more analytical advantages because they can more clearly reflect the brightness information of objects.

[0003] Traditional color cameras are arranged in a Bayer pattern. The color images they generate usually need to go through an interpolation algorithm to obtain some pixel information, resulting in the following major technical problems when converting images to grayscale images:

[0004] 1. Bayer pattern interpolation leads to blurred details

[0005] The traditional Bayer pattern generates color information of some pixels through interpolation algorithms, but this interpolation process can cause image details to be blurred, especially in high-frequency areas (such as edges and textures), affecting the quality after conversion to grayscale images.

[0006] 2. Noise cannot be effectively removed

[0007] In images taken by color cameras, noise is distributed in different color channels, which makes the noise problem more prominent during the conversion to grayscale images, especially in the dark and detailed areas of the image, where noise cannot be completely removed.

[0008] 3. Traditional color-to-grayscale algorithms do not work well

[0009] Traditional color-to-grayscale methods convert color information into grayscale information by simply weighting the RGB channels. This method cannot fully utilize the subtle color changes in color images, resulting in low-quality grayscale images with unclear details.

[0010] 4. The color information in the color image cannot be effectively converted into grayscale information

[0011] Color images contain rich color information, while grayscale images only reflect brightness information. How to efficiently and accurately convert color information into brightness information of grayscale images is a key technical problem that is difficult to solve with traditional methods.

[0012] The information disclosed in this Background Art is only for understanding the background of the inventive concept and therefore it may include information that does not constitute the prior art. Summary of the invention

[0013] In response to the above problem or one of the above problems, an object of the present invention is to provide a color image grayscale conversion method, system, device and medium based on quantitative perceptual loss. Based on a deep learning algorithm and according to image feature information, a reference grayscale image is optimized to generate a grayscale optimized image for characterizing the brightness information and structure of a scene to ensure the correctness of the relationship between brightness and pixels; then the quantitative perceptual loss algorithm is used to map the grayscale optimized image to a high-dimensional latent space, extract high-level perceptual features of the color camera image and the grayscale optimized image, and generate a target grayscale image, so as to better capture the high-level features of the image and further enhance the image detail retention capability, especially focusing on optimizing high-frequency areas such as complex structures and textures, so that the generated grayscale image has a more natural visual effect and rich details.

[0014] In response to the above problem or one of the above problems, the second object of the present invention is to provide a color image grayscale conversion system based on quantitative perception, by setting an image acquisition and processing module, an image feature extraction module, an image optimization processing module and an image quality perception module; first, a basic grayscale image is generated to ensure the correctness of the relationship between brightness and pixels; then, the image detail retention ability is further improved through a quantitative perception loss algorithm and a deep learning algorithm, especially focusing on optimizing high-frequency areas such as complex structures and textures, which can better capture the high-level features of the image, so that the generated grayscale image has a more natural visual effect and rich details.

[0015] In response to the above problem or one of the above problems, the third object of the present invention is to provide a color image grayscale conversion method, system, device and medium based on quantitative perception, taking the denoised image as the target image for deep learning, thereby guiding the network to learn how to effectively remove noise; through this learning mechanism based on deep learning, it is possible to automatically learn the distribution law of noise in different channels, and then perform adaptive noise processing, and at the same time, through quantitative perceptual loss, it is possible to better capture the high-level features of the grayscale image, so that the newly generated grayscale image performs better in terms of structure, texture and other perceptual levels.

[0016] To achieve one of the above purposes, the first technical solution of the present invention is:

[0017] A color image grayscale conversion method based on quantization perceptual loss includes the following steps:

[0018] Step 1: Obtain a color camera image and a reference grayscale image of the target scene through a pre-built image acquisition and processing model;

[0019] Step 2: Using a pre-built image feature extraction model, perform channel decomposition and feature extraction on the color camera image, and fuse the features to obtain fused image feature information;

[0020] Step 3: Using a pre-built image optimization processing model, based on a deep learning algorithm and according to image feature information, the reference grayscale image is optimized to generate a grayscale optimized image that expresses the brightness information and structure of the target scene.

[0021] Step 4: Based on the pre-built image perception optimization model, the quantized perceptual loss algorithm is used to map the color camera image and the grayscale optimized image to a high-dimensional latent space, extract the high-level perceptual features of the color camera image and the grayscale optimized image, and generate the final target grayscale image, thereby completing the conversion of the color image to the grayscale image.

[0022] After continuous exploration and experiments, the present invention constructs an image acquisition and processing model, an image feature extraction model, an image optimization processing model and an image perception optimization model, and optimizes the reference grayscale image based on a deep learning algorithm and according to the image feature information to generate a grayscale optimized image for characterizing the brightness information and structure of a scene to ensure the correctness of the relationship between brightness and pixels; and then uses a quantized perceptual loss algorithm to map the grayscale optimized image to a high-dimensional latent space, extract high-level perceptual features of the color camera image and the grayscale optimized image, and generate a target grayscale image, so as to better capture the high-level features of the image and further enhance the image's detail retention capability, especially focusing on optimizing high-frequency areas such as complex structures and textures, so that the generated grayscale image has a more natural visual effect and rich details.

[0023] Furthermore, the present invention solves the noise problem through an image optimization processing model, and uses the grayscale image after DCT denoising as the target image for deep learning, thereby guiding the network to learn how to effectively remove noise; through this learning mechanism based on deep learning, the network can automatically learn the distribution law of noise in different channels, and then perform adaptive noise processing;

[0024] Furthermore, the present invention calculates the quantized perceptual loss (QPL) of the grayscale optimized image through the image perception optimization model, so as to better capture the high-level features of the grayscale image. Compared with the traditional pixel-level loss optimization, the grayscale image generated by the present invention performs better in terms of structure, texture and other perceptual aspects.

[0025] As the preferred technical measures:

[0026] The reference grayscale image is generated from a grayscale camera image of an unrelated scene;

[0027] Or, the reference grayscale image is generated by a grayscale camera image of the same scene;

[0028] Alternatively, the reference grayscale image is generated by a grayscale camera image of the target scene, and the generation method is as follows:

[0029] Step 1: Obtain a color camera image captured by a color camera according to the Bayer pattern BGGR arrangement, where each pixel in the color camera image records the intensity value of one of the three primary colors: red, green, and blue;

[0030] Obtain a grayscale camera image Gray corresponding to the position of the color camera image, where the grayscale camera image is used to capture the brightness information of the target scene;

[0031] In order to ensure that the color camera image and the grayscale camera image correspond in space, the enhanced correlation coefficient ECC algorithm is used to align the color camera image and the grayscale camera image to generate an aligned grayscale aligned image.

[0032] The enhanced correlation coefficient ECC algorithm calculates the two-dimensional transformation matrix between the images and performs translation, rotation, and scaling operations on the grayscale camera image to ensure the spatial consistency of the color camera image and the grayscale camera image. The image alignment formula based on the strong correlation coefficient ECC algorithm is as follows:

[0033]

[0034] Where I1(x, y) is the grayscale image converted from the color camera image, I2(x, y) is the grayscale camera image captured by the grayscale camera, and μ1 and μ2 are their means;

[0035] Since the length and width of the color camera image Raw of the color camera become half of the original size after Bayer pattern channel processing, the grayscale alignment image needs to be downsampled to match the size of the color camera image; in order to maintain image details and avoid blurring caused by interpolation, the nearest neighbor interpolation is used for downsampling to obtain a grayscale sampling map, so that the grayscale sampling image can match the color camera image Raw in resolution;

[0036] The downsampling formula for the nearest neighbor interpolation is:

[0037] I down (x′,y′)=I(x,y)

[0038] Where (x′, y′) is the pixel coordinate after downsampling, and (x, y) is the pixel coordinate in the grayscale aligned image closest to (x′, y′);

[0039] In order to improve the clarity of the grayscale image and reduce noise interference, discrete cosine transform (DCT) is used to denoise the grayscale sampling image to obtain the final reference grayscale image.

[0040] As the preferred technical measures:

[0041] The method of using discrete cosine transform DCT to denoise the grayscale sampling image and obtain the final reference grayscale image ref2 is as follows:

[0042] Step 11, obtain grayscale sampling image I gray (x,y), whose size is M*M, and the grayscale sampling image I gray (x,y) is divided into N*N sub-blocks;

[0043] Step 12: Calculate the variance σ for each sub-block 2 , and its calculation formula is as follows:

[0044]

[0045] Among them, μ is the mean value of the sub-block, and its calculation formula is as follows:

[0046]

[0047] Step 13: Select the top FS% of sub-blocks with the smallest variance and calculate the mean of their variances as the reference noise variance The calculation formula is as follows:

[0048]

[0049] Where K is the number of selected sub-blocks, i.e., FS% of the total number of image blocks;

[0050] Step 14, performing discrete cosine transform DCT transform on each sub-block to obtain a coefficient matrix C(u,v);

[0051] According to the reference noise variance and an adaptive threshold λ, the coefficient matrix is ​​thresholded to obtain a new coefficient matrix The calculation formula is as follows:

[0052]

[0053] Step 15: After completing the thresholding process, use the inverse discrete cosine transform DCT to transform the new coefficient matrix Convert back to the image domain to obtain the denoised image sub-block;

[0054] Step 16: All image sub-blocks are weighted averaged and concatenated to obtain a final reference grayscale image.

[0055] As the preferred technical measures:

[0056] Step 2: Use the pre-built image feature extraction model to perform channel decomposition and feature extraction on the color camera image, and fuse the features to obtain the fused image feature information as follows:

[0057] Step 21, decomposing the color camera image into four channels according to the Bayer pattern RGGB, including a red channel R, a green channel one G1, a green channel two G2 and a blue channel B;

[0058] Step 22, the red channel R, the green channel 1 G1, the green channel 2 G2 and the blue channel B are respectively subjected to feature extraction by a convolutional neural network CNN; at least three feature maps are obtained for each channel, which include feature Figure 1 Fr, Features Figure 2 Fg and feature map three Fb;

[0059] Step 23: The features of the four channels Figure 1 Fr, Features Figure 2 Fg is fused with feature map 3 Fb to generate image feature information.

[0060] As the preferred technical measures:

[0061] Step 3: Use the pre-built image optimization processing model, based on the deep learning algorithm, and optimize the reference grayscale image according to the image feature information. The method for generating the grayscale optimized image is as follows:

[0062] Based on the image feature information and the reference grayscale image, the first step image loss L1 is calculated for both, which is used for pixel-level supervision. The calculation formula is as follows:

[0063] L1 step1 =|I gen (x,y)-ref2(x,y)|1

[0064] According to the first step image loss, generate a preliminary grayscale image;

[0065] Based on the preliminary grayscale image, a grayscale optimized image is generated through the network structure to represent the brightness information and structure of a scene;

[0066] Calculate grayscale optimized image I final The second-step image loss between (x, y) and the reference grayscale image is used to measure the error of the grayscale optimized image at the pixel level. The calculation formula is as follows:

[0067] L1 step2 =|I final (x,y)-ref2(x,y)|1.

[0068] As the preferred technical measures:

[0069] Step 4: Based on the pre-built image perception optimization model, the color camera image and the grayscale optimized image are mapped to a high-dimensional latent space using a quantized perceptual loss algorithm, and the high-level perceptual features of the color camera image and the grayscale optimized image are extracted to generate the final target grayscale image as follows:

[0070] Step 41: grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are encoded by the quantized perceptual loss algorithm to convert the grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are mapped into a high-dimensional latent space to generate feature variables in the high-dimensional latent space, which include the stage feature variable z final and the reference characteristic variable z ref ; The calculation formula is as follows:

[0071] z final =E(I final (x,y)

[0072] z ref =E(ref2(x,y))

[0073] The feature variables in the high-dimensional latent space are obtained through the encoder of VQ-VAE;

[0074] Step 42, according to the stage characteristic variable z final and the reference characteristic variable z ref , calculate the grayscale optimized image I final The perceptual loss of (x, y) and the reference grayscale image in the high-dimensional latent space is calculated as follows:

[0075] LQPL=|z final -z ref |1

[0076] This perceptual loss is used to measure the similarity between the grayscale optimized image and the reference grayscale image in terms of high-level perceptual features;

[0077] Step 43, weighted coupling is performed on the first step image loss, the second step image loss and the perceptual loss to obtain a total loss function, which is calculated as follows:

[0078]

[0079] Among them: α is a weighting coefficient, which is a weight coefficient between 0 and 1, used to control the relative importance of the first step image loss and the second step image loss in the total loss.

[0080] Step 44, extracting high-level perceptual features of the color camera image and the grayscale optimized image according to the total loss function, and generating a target grayscale image; the high-level perceptual features include at least structural features and texture features.

[0081] To achieve one of the above purposes, the second technical solution of the present invention is:

[0082] A color image grayscale conversion method based on quantization perceptual loss, including the following contents:

[0083] Align and denoise the grayscale camera image to obtain a reference grayscale image;

[0084] Decompose and fuse the features of the color camera image to obtain the fused image feature information;

[0085] Based on the deep learning algorithm and according to the image feature information, the reference grayscale image is optimized to generate a grayscale optimized image;

[0086] The grayscale optimized image is corrected by using the quantization perceptual loss algorithm to generate the target grayscale image, thus realizing the grayscale conversion of color images based on quantization perceptual loss.

[0087] After continuous exploration and experiments, the present invention optimizes the reference grayscale image based on a deep learning algorithm and according to image feature information to generate a grayscale optimized image, which is used to characterize the brightness information and structure of a scene and ensure the correctness of the relationship between brightness and pixels; then uses a quantized perceptual loss algorithm to map the grayscale optimized image to a high-dimensional latent space, extracts high-level perceptual features of the color camera image and the grayscale optimized image, and generates a target grayscale image, so as to better capture the high-level features of the image and further enhance the image's detail retention capability, especially focusing on optimizing high-frequency areas such as complex structures and textures, so that the generated grayscale image has a more natural visual effect and richer details.

[0088] To achieve one of the above purposes, the third technical solution of the present invention is:

[0089] A color image grayscale conversion system based on quantized perceptual loss, comprising an image acquisition processing module, an image feature extraction module, an image optimization processing module and an image quality perception module;

[0090] An image acquisition and processing module is used to obtain a color camera image of a scene and a grayscale camera image of a corresponding position, and align and denoise the grayscale camera image based on the color camera image to obtain a reference grayscale image;

[0091] An image feature extraction module is used to decompose and fuse the features of the color camera image to obtain the fused image feature information;

[0092] The image optimization processing module is used to optimize the reference grayscale image based on the deep learning algorithm and according to the image feature information to generate a grayscale optimized image for representing the brightness information and structure of a scene;

[0093] The image quality perception module is used to map the grayscale optimized image to a high-dimensional latent space using a quantization-perceptual loss algorithm, extract high-level perceptual features of the color camera image and the grayscale optimized image, and generate a target grayscale image, thereby realizing grayscale conversion of color images based on quantization-perceptual loss.

[0094] After continuous exploration and experimentation, the present invention sets an image acquisition and processing module, an image feature extraction module, an image optimization processing module and an image quality perception module; firstly, a basic grayscale image is generated to ensure the correctness of the relationship between brightness and pixels; then, a quantized perceptual loss algorithm and a deep learning algorithm are used to further enhance the detail retention capability of the image, especially focusing on optimizing high-frequency areas such as complex structures and textures, which can better capture the high-level features of the image, making the generated grayscale image more natural in visual effect and rich in details.

[0095] At the same time, the present invention uses the denoised image as the target image for deep learning, thereby guiding the network to learn how to effectively remove noise; through this learning mechanism based on deep learning, it is possible to automatically learn the distribution laws of noise in different channels, and then perform adaptive noise processing. The solution is scientific, reasonable and feasible.

[0096] To achieve one of the above purposes, the fourth technical solution of the present invention is:

[0097] An electronic device comprising:

[0098] one or more processors;

[0099] A storage device for storing one or more programs;

[0100] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned color image grayscale conversion method based on quantization perception loss.

[0101] To achieve one of the above purposes, the fifth technical solution of the present invention is:

[0102] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned color image grayscale conversion method based on quantization perception loss.

[0103] Compared with the prior art solutions, the present invention has the following beneficial effects:

[0104] After continuous exploration and experiments, the present invention optimizes the reference grayscale image based on a deep learning algorithm and according to image feature information to generate a grayscale optimized image, which is used to characterize the brightness information and structure of a scene and ensure the correctness of the relationship between brightness and pixels; then uses a quantized perceptual loss algorithm to map the grayscale optimized image to a high-dimensional latent space, extracts high-level perceptual features of the color camera image and the grayscale optimized image, and generates a target grayscale image, so as to better capture the high-level features of the image and further enhance the image's detail retention capability, especially focusing on optimizing high-frequency areas such as complex structures and textures, so that the generated grayscale image has a more natural visual effect and richer details.

[0105] After continuous exploration and experimentation, the present invention sets an image acquisition and processing module, an image feature extraction module, an image optimization processing module and an image quality perception module; firstly, a basic grayscale image is generated to ensure the correctness of the relationship between brightness and pixels; then, a quantized perceptual loss algorithm and a deep learning algorithm are used to further enhance the detail retention capability of the image, especially focusing on optimizing high-frequency areas such as complex structures and textures, which can better capture the high-level features of the image, making the generated grayscale image more natural in visual effect and rich in details.

[0106] At the same time, the present invention uses the denoised image as the target image for deep learning, thereby guiding the network to learn how to effectively remove noise; through this learning mechanism based on deep learning, it is possible to automatically learn the distribution laws of noise in different channels, and then perform adaptive noise processing. The solution is scientific, reasonable and feasible.

[0107] Furthermore, the present invention can better capture the high-level features of grayscale images by using quantized perceptual loss (QPL). Compared with traditional pixel-level loss optimization, the grayscale images generated by the present invention have better performance in terms of structure, texture and other perceptual aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 A schematic diagram of a process of a color image grayscale conversion method of the present invention;

[0109] Figure 2 Another flowchart of the color image grayscale conversion method of the present invention is shown in FIG. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0111] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. Those skilled in the art can fully understand the present invention without the description of these details.

[0112] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0113] The first specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention is as follows:

[0114] A color image grayscale conversion method based on quantization perceptual loss includes the following steps:

[0115] Step 1: Obtain a color camera image and a reference grayscale image of the target scene through a pre-built image acquisition and processing model;

[0116] Step 2: Using a pre-built image feature extraction model, perform channel decomposition and feature extraction on the color camera image, and fuse the features to obtain fused image feature information;

[0117] Step 3: Using a pre-built image optimization processing model, based on a deep learning algorithm and according to image feature information, the reference grayscale image is optimized to generate a grayscale optimized image that expresses the brightness information and structure of the target scene.

[0118] Step 4: Based on the pre-built image perception optimization model, the quantized perceptual loss algorithm is used to map the color camera image and the grayscale optimized image to a high-dimensional latent space, extract the high-level perceptual features of the color camera image and the grayscale optimized image, and generate the final target grayscale image, thereby completing the conversion of the color image to the grayscale image.

[0119] A second specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention:

[0120] A color image grayscale conversion method based on quantization perceptual loss, characterized by:

[0121] The following steps are involved:

[0122] Step 1: Obtain the color camera image Raw of the target scene and the grayscale camera image Gray of the corresponding position through the pre-built image acquisition and processing model, and align and denoise the grayscale camera images to generate a reference grayscale image;

[0123] Step 2: Using the pre-built image feature extraction model, perform channel decomposition and feature extraction on the Raw image of the color camera, and fuse the features to obtain fused image feature information;

[0124] Step 3: Using a deep learning-based image optimization processing model, based on the fused image feature information, the reference grayscale image is deeply optimized to generate a grayscale optimized image that expresses the brightness information and structure of the target scene.

[0125] Step 4: Based on the pre-built image perception optimization model, the quantized perceptual loss algorithm is used to map the color camera image and the grayscale optimized image to a high-dimensional latent space, extract the high-level perceptual features of the color camera image and the grayscale optimized image, and generate the final target grayscale image, thereby completing the conversion of the color image to the grayscale image.

[0126] like Figure 1 As shown, the third specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention is as follows:

[0127] A color image grayscale conversion method based on quantization perceptual loss includes the following steps:

[0128] Step 1: Obtain a color camera image of a scene and a grayscale camera image of a corresponding position through a pre-built image acquisition and processing model, and align and denoise the grayscale camera image based on the color camera image to obtain a reference grayscale image;

[0129] Step 2: Using a pre-built image feature extraction model, the color camera image is decomposed and feature-fused to obtain fused image feature information;

[0130] Step 3: Using a pre-built image optimization processing model, based on a deep learning algorithm and according to image feature information, the reference grayscale image is optimized to generate a grayscale optimized image for representing the brightness information and structure of a scene;

[0131] Step 4: Based on the pre-built image perception optimization model, the color RAW image is mapped to a high-dimensional latent space using the quantization perceptual loss algorithm, the high-level perceptual features of the image are extracted, and the target grayscale image is generated to achieve color image grayscale conversion based on quantization perceptual loss.

[0132] In this embodiment: Step 1, a color camera image of a scene and a grayscale camera image of a corresponding position are obtained through a pre-built image acquisition and processing model, and the grayscale camera image is aligned and denoised based on the color camera image to obtain a reference grayscale image as follows:

[0133] Get the color camera image Raw taken by the color camera of a certain scene;

[0134] The color camera image Raw is arranged in pixels according to the Bayer pattern BGGR, and each pixel only records the intensity value of one of the three primary colors: red, green, or blue;

[0135] According to the color camera image, obtain the grayscale camera image Gray taken by the grayscale camera at the same position as the color camera image; the grayscale camera image is used to obtain the brightness information in a scene;

[0136] In order to ensure the spatial correspondence between the color camera image and the grayscale image, the enhanced correlation coefficient (ECC) algorithm is used to align the color camera image and the grayscale image to obtain a grayscale aligned image.

[0137] The enhanced correlation coefficient (ECC) algorithm can calculate the two-dimensional transformation matrix between two images and perform translation, rotation and scaling of the grayscale image, thereby aligning the color camera image with the grayscale image.

[0138] The image alignment formula based on the ECC algorithm is as follows:

[0139]

[0140] Where I1(x, y) is the grayscale image converted from the color camera image, I2(x, y) is the grayscale image captured by the grayscale camera, and μ1 and μ2 are their means;

[0141] Since the length and width of the color camera image Raw of the color camera become half of the original size after Bayer pattern channel processing, the grayscale alignment image needs to be downsampled to match the size of the color camera image; in order to maintain image details and avoid blurring caused by interpolation, the nearest neighbor interpolation is used for downsampling to obtain a grayscale sampling map, so that the grayscale sampling image can match the color camera image Raw in resolution;

[0142] The downsampling formula for the nearest neighbor interpolation is:

[0143] I down (x′,y′)=I(x,y)

[0144] Where (x′, y′) is the pixel coordinate after downsampling, and (x, y) is the pixel coordinate in the grayscale aligned image closest to (x′, y′);

[0145] In order to improve the clarity of the grayscale image and reduce noise interference, discrete cosine transform (DCT) is used to denoise the grayscale sampling image to obtain the final reference grayscale image.

[0146] In this embodiment, the method of using discrete cosine transform DCT to perform denoising on the grayscale sampling image to obtain the final reference grayscale image ref2 is as follows:

[0147] Step 11, obtain grayscale sampling image I gray (x,y), whose size is M*M, and the grayscale sampling image I gray (x,y) is divided into N*N sub-blocks;

[0148] Step 12: Calculate the variance σ for each sub-block 2 , and its calculation formula is as follows:

[0149]

[0150] Among them, μ is the mean value of the sub-block, and its calculation formula is as follows:

[0151]

[0152] Step 13: Select the top FS% of sub-blocks with the smallest variance and calculate the mean of their variances as the reference noise variance The calculation formula is as follows:

[0153]

[0154] Where K is the number of selected sub-blocks, i.e., FS% of the total number of image blocks;

[0155] Step 14, performing discrete cosine transform DCT transform on each sub-block to obtain a coefficient matrix C(u,v);

[0156] According to the reference noise variance and an adaptive threshold λ, the coefficient matrix is ​​thresholded to obtain a new coefficient matrix The calculation formula is as follows:

[0157]

[0158] Step 15: After completing the thresholding process, use the inverse discrete cosine transform DCT to transform the new coefficient matrix Convert back to the image domain to obtain the denoised image sub-block;

[0159] Step 16: All image sub-blocks are weighted averaged and concatenated to obtain a final reference grayscale image.

[0160] In this embodiment: Step 2, using a pre-built image feature extraction model to decompose and fuse features of a color camera image, a method for obtaining fused image feature information is as follows:

[0161] Step 21, decomposing the color camera image into four channels according to the Bayer pattern RGGB, including a red channel R, a green channel one G1, a green channel two G2 and a blue channel B;

[0162] Step 22, the red channel R, the green channel 1 G1, the green channel 2 G2 and the blue channel B are respectively subjected to feature extraction by a convolutional neural network CNN; at least three feature maps are obtained for each channel, which include feature Figure 1 Fr, Features Figure 2 Fg and feature map three Fb;

[0163] Step 23: The features of the four channels Figure 1 Fr, Features Figure 2 Fg is fused with feature map 3 Fb to generate image feature information.

[0164] In this embodiment: Step 3, a method for optimizing a reference grayscale image by using a pre-built image optimization processing model based on a deep learning algorithm and according to image feature information to generate a grayscale optimized image is as follows:

[0165] Based on the image feature information and the reference grayscale image, the first step image loss L1 is calculated for both, which is used for pixel-level supervision. The calculation formula is as follows:

[0166] L1 step1 =|I gen (x,y)-ref2(x,y)|1

[0167] According to the first step image loss, generate a preliminary grayscale image;

[0168] Based on the preliminary grayscale image, a grayscale optimized image is generated through the network structure to represent the brightness information and structure of a scene;

[0169] Calculate grayscale optimized image I final The second-step image loss between (x, y) and the reference grayscale image is used to measure the error of the grayscale optimized image at the pixel level. The calculation formula is as follows:

[0170] L1 step2 =|I final (x,y)-ref2(x,y)|1.

[0171] In this embodiment: Step 4, based on the pre-built image perception optimization model, using the quantized perception loss algorithm, the grayscale optimized image is mapped to a high-dimensional latent space, the high-level perception features of the image are extracted, and the method of generating the target grayscale image is as follows:

[0172] Step 41: grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are encoded by the quantized perceptual loss algorithm to convert the grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are mapped into a high-dimensional latent space to generate feature variables in the high-dimensional latent space, which include the stage feature variable z final and the reference characteristic variable z ref ; The calculation formula is as follows:

[0173] z final =E(I final (x,y)

[0174] z ref =E(ref2(x,y))

[0175] The quantization perceptual loss algorithm is the encoder VQ-VAE;

[0176] Step 42, according to the stage characteristic variable z final and the reference characteristic variable z ref , calculate the grayscale optimized image I final The perceptual loss of (x, y) and the reference grayscale image in the high-dimensional latent space is calculated as follows:

[0177] LQPL=|z final -z ref |1

[0178] This perceptual loss is used to measure the similarity between the grayscale optimized image and the reference grayscale image in terms of high-level perceptual features;

[0179] Step 43, weighted coupling is performed on the first step image loss, the second step image loss and the perceptual loss to obtain a total loss function, which is calculated as follows:

[0180]

[0181] Among them: α is a weighting coefficient, which is a weight coefficient between 0 and 1, used to control the relative importance of the first step image loss and the second step image loss in the total loss.

[0182] Step 44, extracting high-level perceptual features of the color camera image and the grayscale optimized image according to the total loss function, and generating a target grayscale image;

[0183] High-level perceptual features include at least structural features and texture features.

[0184] A fourth specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention:

[0185] A color image grayscale conversion method based on quantization perceptual loss, including the following contents:

[0186] Obtain a color camera image of a scene and a grayscale camera image of a corresponding position, and align and denoise the grayscale camera image based on the color camera image to obtain a reference grayscale image;

[0187] Decompose and fuse the features of the color camera image to obtain the fused image feature information;

[0188] Based on the deep learning algorithm and according to the image feature information, the reference grayscale image is optimized to generate a grayscale optimized image;

[0189] The color RAW image is corrected by using the quantization perceptual loss algorithm to generate the target grayscale image, thus realizing the grayscale conversion of the color image based on quantization perceptual loss.

[0190] like Figure 2 As shown, the fifth specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention is as follows:

[0191] A color image grayscale conversion method based on quantized perceptual loss is a color to grayscale conversion method based on the alignment of color camera Raw images and grayscale camera images, denoising and deep learning. It can generate high-quality grayscale images close to the effect of grayscale cameras. Through image alignment, DCT adaptive denoising, multi-channel feature fusion and deep learning network optimization, efficient image denoising and detail retention are achieved. It includes the following steps:

[0192] Step 1: Align the color image with the grayscale image, which includes the following:

[0193] Get the original Raw image of the color camera and the grayscale camera image Gray at the corresponding position of the color image. The Raw image of the color camera is usually arranged according to the Bayer pattern (BGGR), and each pixel only records the intensity value of one of the three primary colors of red, green or blue; while the grayscale camera image obtains the brightness information in the scene.

[0194] Furthermore, to ensure the spatial correspondence between the color image and the grayscale image, the enhanced correlation coefficient (ECC) algorithm is used to align the images taken by the color camera and the grayscale camera. The ECC algorithm can calculate the two-dimensional transformation matrix (including translation, rotation, and scaling) between the two images, thereby achieving accurate image alignment.

[0195] The ECC algorithm is the EnhancedCorrelationCoefficient algorithm, and its image alignment formula is as follows:

[0196]

[0197] Among them, ECC is the correlation between images, I1(x, y) and I2(x, y) are the color-to-grayscale image and the grayscale image captured by the grayscale camera, respectively, and μ1 and μ2 are their means.

[0198] Step 2: Downsampling and adaptive DCT denoising of grayscale images are as follows:

[0199] Since the raw image of the color camera is processed by Bayer pattern channel division, the length and width of the image become half of the original size, so the grayscale image needs to be downsampled to match the size of the color image. In order to maintain image details and avoid blurring caused by interpolation, the nearest neighbor interpolation is used for downsampling.

[0200] The downsampling formula for the nearest neighbor interpolation is:

[0201] I down (x′,y′)=I(x,y)

[0202] Among them I down is the downsampled image, (x′, y′) is the pixel coordinate after downsampling, and (x, y) is the pixel coordinate in the original grayscale image closest to (x′, y′). Therefore, the downsampled reference grayscale image ref1 can match the color Raw image in resolution.

[0203] In order to further improve the clarity of grayscale images and reduce noise interference, discrete cosine transform (DCT) is used to denoise grayscale images. DCT denoising has good localized frequency domain processing capabilities and can effectively remove noise in images. The specific steps are as follows:

[0204] Step 21, divide the downsampled grayscale image into N*N sub-blocks, and process each sub-block separately. gray (x,y), size is M*M

[0205] Step 22, calculate the variance σ for each sub-block 2 , the formula is:

[0206]

[0207] Among them, μ is the mean value of the sub-block:

[0208]

[0209] Step 23: Select the top 10% of image blocks with the smallest variance and calculate the mean of their variance as the reference noise variance. Defined as:

[0210]

[0211] Wherein, K is the number of selected image blocks, that is, 10% of the total number of image blocks.

[0212] Step 24, perform DCT denoising transformation on each sub-block to obtain a coefficient matrix C(u,v). Perform thresholding on the DCT coefficients based on the estimated noise variance and an adaptive threshold λ:

[0213]

[0214] After completing the thresholding process, use the inverse DCT to transform the processed coefficient matrix Convert back to the image domain to get the denoised image block. All sub-blocks are weighted averaged and concatenated to get the final reference grayscale image ref2.

[0215] Step 3: The method for decomposing the color Raw image and extracting multi-channel features is as follows:

[0216] Step 31. Decompose the color Raw image into four channels according to the Bayer pattern (RGGB): R (red channel), G1 (green channel one), G2 (green channel two), and B (blue channel).

[0217] Step 32. For each channel (R, G1, G2 and B), feature extraction is performed through a convolutional neural network (CNN) to obtain three feature maps Fr, Fg and Fb.

[0218] Step 33. Fuse the extracted four channel features Fr, Fg and Fb to generate a fused feature map Fbgr. The fused feature map will be used to generate a grayscale image.

[0219] Step 4: The method of color image grayscale conversion and loss optimization based on deep learning is as follows:

[0220] The first step is to generate the image I gen (x, y) and the denoised reference grayscale image ref2(x, y) calculate the L1 loss for pixel-level supervision, the formula is as follows:

[0221] L1 step1 =|I gen (x,y)-ref2(x,y)|1

[0222] This loss is only used for supervised learning during the training phase to help the network generate preliminary grayscale images. This step is not performed during inference.

[0223] The second step is to generate the final grayscale image through a deeper network structure (such as Unet network), denoted as I final (x,y). During inference, the network only outputs this image. And calculates the final generated I final The L1 loss of (x, y) and the reference image ref2(x, y) is used to measure the error of the final image at the pixel level. The formula is as follows:

[0224] L1 step2 =|I final (x,y)-ref2(x,y)|1

[0225] During the training phase, the network generates two images and calculates different loss functions at the same time: the image generated in the first step is used for supervision through L1 loss to help the network learn how to generate basic grayscale images; and the final image is optimized using a weighted combination of Quantized Perception Loss (QPL) and L1 loss. During inference, the network only outputs the final high-quality grayscale image.

[0226] Step 5. In order to improve the perceptual quality of the final image, the VQ-VAE encoding module is used to calculate the quantization perceptual loss (QPL). The specific steps are as follows:

[0227] The generated grayscale image I final (x, y) and the reference grayscale image ref2(x, y) are encoded by the VQ-VAE encoder to generate the feature representation z in the latent space final and z ref , and its calculation formula is as follows:

[0228] z final =E(I final (x,y)

[0229] z ref =E(ref2(x,y))

[0230] Calculate the perceptual loss of the two in the latent space as follows:

[0231] LQPL=|z final -z ref |1

[0232] This loss is used to measure the similarity of the final generated grayscale image in terms of high-order features such as structure and texture.

[0233] The final total loss function is a weighted combination of the three losses, and the specific formula is as follows:

[0234]

[0235] Where: α is a weighting coefficient, which is a weighting coefficient between 0 and 1, used to control the relative importance of the first step loss and the second step loss in the total loss. In this embodiment, α is taken as 0.4, L1 step1 is the L1 loss of the first step to generate the image, which is used to supervise the generation of the basic grayscale image. step2 It is the L1 loss of the second step to generate the image, which is used to optimize the brightness and pixel-level differences of the final grayscale image. When generating the image in the second step, the quantized perceptual loss (QPL) optimizes the image through the differences in the high-dimensional feature space, rather than relying on traditional pixel-level losses (such as L1 or L2 losses) for pixel-by-pixel comparison. Traditional losses often ignore high-level features in the image (such as edges, textures, structures, etc.), resulting in poor perceptual quality of the generated image. QPL focuses on the optimization of these high-level features, making the generated grayscale image closer to the reference image in the high-level feature space. Compared with pixel-level optimization, QPL can better capture the structure and texture of the image, so that the generated image not only has accurate brightness information, but also is more visually natural and rich in details, which is in line with the human eye's perception of structure and texture.

[0236] This embodiment is a color camera grayscale conversion method based on VQ (Vector Quantization) perception, which constructs a color image to grayscale framework based on VQ perception optimization. By introducing the VQ-VAE (Vector Quantization Variational Autoencoder) module in the color image grayscale conversion process, the perceived quality of the generated image is significantly improved. At the same time, the framework uses VQ Encode Block to encode the generated grayscale image and the reference image, and maps the image to a high-dimensional latent space, so that the grayscale optimizes the structure and texture features of the image by calculating the difference between the latent vectors. This framework solves the problem of detail loss and poor perceived quality that is easily caused in the traditional color image to grayscale conversion process.

[0237] At the same time, this embodiment adopts a multi-stage generation structure, which is divided into two steps to gradually optimize the generation results. At the same time, a step-by-step optimization strategy is proposed. The basic grayscale image is generated in the first stage to provide basic brightness information and structure. The second stage further optimizes the generated final grayscale image and performs more advanced optimization at the pixel-level accuracy and perception level. This framework effectively solves the common problems of image details and edge blur in the process of converting color images to grayscale, making the generated image clearer and more natural in structure and perception level.

[0238] A sixth specific embodiment of the color image grayscale conversion method based on quantized perceptual loss of the present invention:

[0239] A color image grayscale conversion method based on quantization perceptual loss, including the following contents:

[0240] Through a multi-stage generation architecture, a basic grayscale image is first generated to ensure the correctness of the relationship between brightness and pixels; then, VQ perception optimization and deep learning networks are used to further improve the image's detail retention capabilities, especially focusing on optimizing high-frequency areas such as complex structures and textures.

[0241] Traditional color image to grayscale conversion methods, especially in Bayer pattern-based color sensors, often cause detail loss due to interpolation, especially in edges and high-frequency areas, where image blur is very obvious. Through a multi-stage generation architecture, the problem of detail loss and blur in the process of color image to grayscale conversion can be effectively solved.

[0242] The noise problem is solved by using a deep learning model, and the grayscale image after DCT denoising is used as the target image for deep learning, thereby guiding the network to learn how to effectively remove noise. Through this learning mechanism based on deep learning, the network can automatically learn the distribution law of noise in different channels, and then perform adaptive noise processing.

[0243] Since the noise in different channels of the color sensor is unevenly distributed, it is difficult for the traditional color-to-grayscale method to effectively remove the noise, resulting in more noise interference in the generated grayscale image, especially in low-brightness areas. The present invention generates a cleaner grayscale image through deep learning noise processing.

[0244] Through VQ Encode Block, vector quantization perceptual optimization (VQ perceptual optimization) is performed, and quantization perceptual loss (QPL) is introduced, allowing the network to capture high-level feature information (such as structure and texture) of the image in the latent space. Through this high-level feature optimization mechanism, the generated grayscale image can be closer to the reference image in terms of perceptual quality.

[0245] Since traditional color image to grayscale conversion methods mostly rely on pixel-level loss functions (such as L1 or L2 loss), this method tends to ignore the high-level perceptual features of the image, resulting in poor consistency of the generated grayscale image at high levels such as structure and texture, affecting the overall visual effect. The present invention introduces quantized perceptual loss (QPL), which can effectively improve the perceptual quality of the generated image and enhance visual consistency.

[0246] An embodiment of a device applying the method of the present invention:

[0247] An electronic device comprising:

[0248] one or more processors;

[0249] A storage device for storing one or more programs;

[0250] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned color image grayscale conversion method based on quantization perception loss.

[0251] A computer medium embodiment using the method of the present invention:

[0252] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned color image grayscale conversion method based on quantization perception loss.

[0253] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, and computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0254] The present application is described by flowcharts or / and block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each process or / and block in the flowchart or / and block diagram and the combination of the processes or / and blocks in the flowchart or / and block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart or / and block diagram. Figure 1 Process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0255] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0256] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0257] The model in this application is an object that objectively describes the morphological structure with the help of physical or virtual representation. The object is not equal to the object and is not limited to physical and virtual. It can be a data processing function, software program, processing mode, usage method, operation method, workflow, application process, electronic hardware, circuit module, processing system, system imitation or simulation object.

[0258] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field can still modify or replace the specific implementation methods of the present invention with equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A color image grayscale conversion method based on quantization perceptual loss, characterized in that: The following steps are involved: Step 1: Obtain a color camera image and a reference grayscale image of the target scene through a pre-built image acquisition and processing model; Step 2: Using a pre-built image feature extraction model, perform channel decomposition and feature extraction on the color camera image, and fuse the features to obtain fused image feature information; Step 3: Using a pre-built image optimization processing model, based on a deep learning algorithm and according to image feature information, the reference grayscale image is optimized to generate a grayscale optimized image that expresses the brightness information and structure of the target scene. Step 4: Based on the pre-built image perception optimization model, the quantized perceptual loss algorithm is used to map the color camera image and the grayscale optimized image to a high-dimensional latent space, extract the high-level perceptual features of the color camera image and the grayscale optimized image, and generate the final target grayscale image, thereby completing the conversion of the color image to the grayscale image.

2. The method for grayscale conversion of a color image based on quantization perceptual loss according to claim 1, characterized in that: The reference grayscale image is generated from a grayscale camera image of an unrelated scene; Or, the reference grayscale image is generated by a grayscale camera image of the same scene; Alternatively, the reference grayscale image is generated by a grayscale camera image of the target scene, and the generation method is as follows: Step 1: Obtain a color camera image captured by a color camera according to the Bayer pattern BGGR arrangement, where each pixel in the color camera image records the intensity value of one of the three primary colors: red, green, and blue; Obtain a grayscale camera image Gray corresponding to the position of the color camera image, where the grayscale camera image is used to capture the brightness information of the target scene; In order to ensure that the color camera image and the grayscale camera image correspond in space, the enhanced correlation coefficient ECC algorithm is used to align the color camera image and the grayscale camera image to generate an aligned grayscale aligned image. The enhanced correlation coefficient ECC algorithm calculates the two-dimensional transformation matrix between the images and performs translation, rotation, and scaling operations on the grayscale camera image to ensure the spatial consistency of the color camera image and the grayscale camera image. The image alignment formula based on the strong correlation coefficient ECC algorithm is as follows: Where I1(x, y) is the grayscale image converted from the color camera image, I2(x, y) is the grayscale camera image captured by the grayscale camera, and μ1 and μ2 are their means; Since the length and width of the color camera image Raw of the color camera become half of the original size after Bayer pattern channel processing, the grayscale alignment image needs to be downsampled to match the size of the color camera image; in order to maintain image details and avoid blurring caused by interpolation, the nearest neighbor interpolation is used for downsampling to obtain a grayscale sampling map, so that the grayscale sampling image can match the color camera image Raw in resolution; The downsampling formula for the nearest neighbor interpolation is: I down (x′,y′)=I(x,y) Where (x′, y′) is the pixel coordinate after downsampling, and (x, y) is the pixel coordinate in the grayscale aligned image closest to (x′, y′); In order to improve the clarity of the grayscale image and reduce noise interference, discrete cosine transform (DCT) is used to denoise the grayscale sampling image to obtain the final reference grayscale image.

3. The method for grayscale conversion of color images based on quantization perceptual loss according to claim 2, characterized in that: The method of using discrete cosine transform DCT to denoise the grayscale sampling image and obtain the final reference grayscale image ref2 is as follows: Step 11, obtain grayscale sampling image I gray (x,y), whose size is M*M, and the grayscale sampling image I gray (x,y) is divided into N*N sub-blocks; Step 12: Calculate the variance σ for each sub-block 2 , and its calculation formula is as follows: Among them, μ is the mean value of the sub-block, and its calculation formula is as follows: Step 13: Select the top FS% of sub-blocks with the smallest variance and calculate the mean of their variances as the reference noise variance The calculation formula is as follows: Where K is the number of selected sub-blocks, i.e., FS% of the total number of image blocks; Step 14, performing discrete cosine transform DCT transform on each sub-block to obtain a coefficient matrix C(u,v); According to the reference noise variance and an adaptive threshold λ, the coefficient matrix is ​​thresholded to obtain a new coefficient matrix The calculation formula is as follows: Step 15: After completing the thresholding process, use the inverse discrete cosine transform DCT to transform the new coefficient matrix Convert back to the image domain to obtain the denoised image sub-block; Step 16: All image sub-blocks are weighted averaged and concatenated to obtain a final reference grayscale image.

4. The method for grayscale conversion of color images based on quantization perceptual loss according to claim 1, characterized in that: Step 2: Use the pre-built image feature extraction model to perform channel decomposition and feature extraction on the color camera image, and fuse the features to obtain the fused image feature information as follows: Step 21, decomposing the color camera image into four channels according to the Bayer pattern RGGB, including a red channel R, a green channel one G1, a green channel two G2 and a blue channel B; Step 22, performing feature extraction on the red channel R, the green channel 1 G1, the green channel 2 G2 and the blue channel B respectively through the convolutional neural network CNN; at least three feature maps are obtained for each channel, including a feature map 1 Fr, a feature map 2 Fg and a feature map 3 Fb; Step 23, the feature map 1 Fr, the feature map 2 Fg and the feature map 3 Fb of the four channels are fused to generate image feature information.

5. The method for grayscale conversion of color images based on quantization perceptual loss according to claim 1, characterized in that: Step 3: Use the pre-built image optimization processing model, based on the deep learning algorithm, and optimize the reference grayscale image according to the image feature information. The method for generating the grayscale optimized image is as follows: Based on the image feature information and the reference grayscale image, the first step image loss L1 is calculated for both, which is used for pixel-level supervision. The calculation formula is as follows: L1 step1 =|I gen (x,y)-ref2(x,y)|1 According to the first step image loss, generate a preliminary grayscale image; Based on the preliminary grayscale image, a grayscale optimized image is generated through the network structure to represent the brightness information and structure of a scene; Calculate grayscale optimized image I final The second-step image loss between (x, y) and the reference grayscale image is used to measure the error of the grayscale optimized image at the pixel level. The calculation formula is as follows: L1 step2 =|I final (x,y)-ref2(x,y)|1。 6. The method for grayscale conversion of color images based on quantization perceptual loss according to claim 1, characterized in that: Step 4: Based on the pre-built image perception optimization model, the color camera image and the grayscale optimized image are mapped to a high-dimensional latent space using a quantized perceptual loss algorithm, and the high-level perceptual features of the color camera image and the grayscale optimized image are extracted to generate the final target grayscale image as follows: Step 41: grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are encoded by the quantized perceptual loss algorithm to convert the grayscale optimized image I final (x, y) and the reference grayscale image ref2(x, y) are mapped into a high-dimensional latent space to generate feature variables in the high-dimensional latent space, which include the stage feature variable z final and the reference characteristic variable z ref ; The calculation formula is as follows: z final =E(I final (x,y)) from ref =E(ref2(x,y)) The feature variables in the high-dimensional latent space are obtained through the encoder of VQ-VAE; Step 42, according to the stage characteristic variable z final and the reference characteristic variable z ref , calculate the grayscale optimized image I final The perceptual loss of (x,y) and the reference grayscale image in the high-dimensional latent space is calculated as follows: LQPL=|z final -With ref |1 This perceptual loss is used to measure the similarity between the grayscale optimized image and the reference grayscale image in terms of high-level perceptual features; Step 43, weighted coupling is performed on the first step image loss, the second step image loss and the perceptual loss to obtain a total loss function, which is calculated as follows: Among them: α is a weighting coefficient, which is a weight coefficient between 0 and 1, used to control the relative importance of the first step image loss and the second step image loss in the total loss. Step 44, extracting high-level perceptual features of the color camera image and the grayscale optimized image according to the total loss function, and generating a target grayscale image; the high-level perceptual features include at least structural features and texture features.

7. A color image grayscale conversion method based on quantization perceptual loss, characterized in that: Includes the following: Align and denoise the grayscale camera image to obtain a reference grayscale image; Decompose and fuse the features of the color camera image to obtain the fused image feature information; Based on the deep learning algorithm and according to the image feature information, the reference grayscale image is optimized to generate a grayscale optimized image; The grayscale optimized image is corrected by using the quantization perceptual loss algorithm to generate the target grayscale image, thus realizing the grayscale conversion of color images based on quantization perceptual loss.

8. A color image grayscale conversion system based on quantization perceptual loss, characterized in that: It includes an image acquisition and processing module, an image feature extraction module, an image optimization processing module and an image quality perception module; An image acquisition and processing module is used to obtain a color camera image of a scene and a grayscale camera image of a corresponding position, and align and denoise the grayscale camera image based on the color camera image to obtain a reference grayscale image; An image feature extraction module is used to decompose and fuse the features of the color camera image to obtain the fused image feature information; The image optimization processing module is used to optimize the reference grayscale image based on the deep learning algorithm and according to the image feature information to generate a grayscale optimized image for representing the brightness information and structure of a scene; The image quality perception module is used to map the grayscale optimized image to a high-dimensional latent space using a quantization-perceptual loss algorithm, extract high-level perceptual features of the color camera image and the grayscale optimized image, and generate a target grayscale image, thereby realizing grayscale conversion of color images based on quantization-perceptual loss.

9. An electronic device, characterized in that: It includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a color image grayscale conversion method based on quantization perceptual loss as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, a color image grayscale conversion method based on quantization perceptual loss as described in any one of claims 1 to 8 is implemented.

Images