Image conversion method, device, computer equipment and storage medium

Abstract

The present application relates to an image conversion method, device, computer equipment and storage medium. The method comprises: using a bit depth expansion module to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image; using an inverse image processing module to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image. The method can reduce the error between the converted Raw image and the original Raw image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image conversion method, device, computer equipment and storage medium. Background Art

[0002] In the field of image processing, data directly collected by cameras and other image acquisition devices are called Raw images, also known as electronic negatives. Images in this format are a data storage format that is closely related to image acquisition devices.

[0003] Usually, the original Raw image stores the most original data after the sensor collects it. When the collected image data needs to be displayed, the original Raw image needs to be processed by a special Image Signal Processing (ISP) to obtain a standard color channel image that conforms to the human visual system, namely a standard RGB image (standard Red Green Blue, sRGB image for short). Compared with sRGB images, since the original Raw image has a higher bit depth and a higher dynamic range, and the value of the original Raw image is linearly related to the light radiation intensity of the scene, it is often necessary to convert the sRGB image into a Raw image for processing in order to adjust the image style based on the original Raw image in photography, and the convolutional neural network based on the original Raw image has better effects in denoising and super-resolution tasks. The traditional way of converting an sRGB image into a Raw image is usually achieved by the inverse process of ISP.

[0004] However, in the inverse process of ISP, the error between the converted Raw image and the original Raw image is large. Summary of the invention

[0005] Based on this, it is necessary to provide an image conversion method, device, computer equipment and storage medium that can reduce image errors in response to the above technical problems.

[0006] An image conversion method, the method comprising:

[0007] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0008] An inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0009] In one embodiment, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; the bit depth extension module is used to perform bit depth extension processing on the low-bit standard color channel map to obtain a high-bit standard color channel map, including:

[0010] The convolution submodule is used to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map;

[0011] Using the target activation function layer to predict the coefficients of the intermediate low-bit standard color channel image to obtain normalized interpolation coefficients;

[0012] The interpolation submodule is used to process the normalized interpolation coefficient and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

[0013] In one embodiment, the activation function of the target activation function layer is a softmax function; the interpolation submodule includes l pre =α*l max +(1-α)*l min The relationship between

[0014] Among them, l pre is the pixel value of a pixel in the high-bit standard color channel image, l min is the minimum pixel value among the pixels of the low-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, and α and (1-α) are normalized interpolation coefficients.

[0015] In one of the embodiments, the bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes a plurality of pairs of low-bit color training images and corresponding high-bit color training images.

[0016] In one embodiment, the inverse image processing module includes a color conversion submodule and a monotone nonlinear transformation submodule; the inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image, including:

[0017] The color space conversion submodule is used to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix;

[0018] The monotone nonlinear transformation submodule is used to perform nonlinear processing on the high-bit standard color channel image to obtain a linearized linear feature image;

[0019] The linear feature map is transformed using the color space conversion matrix to obtain the high-bit original map.

[0020] In one embodiment, the monotone nonlinear transformation submodule is a module including a monotone function using a Crenshaw-Curtis orthogonal algorithm.

[0021] In one of the embodiments, the inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes a plurality of pairs of high-bit color training images and corresponding high-bit original training images.

[0022] In one embodiment, the method further comprises:

[0023] Performing a target processing operation on the high-level original image to obtain a target high-level original image;

[0024] Using the inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image;

[0025] A quantization operation based on bit depth compression is performed on the target high-bit color channel map to obtain the target low-bit color channel map.

[0026] In one embodiment, the bit depth of the low-bit standard color channel image is 8 bits, and the bit depth of the high-bit original image is 10 bits or 16 bits.

[0027] An image conversion device, comprising:

[0028] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0029] The inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0030] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0032] An inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0034] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0035] An inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0036] The above-mentioned image conversion method, device, computer equipment and storage medium use a bit depth extension module to perform bit depth extension processing on a low-bit standard color channel image to obtain a high-bit standard color channel image, thereby achieving the expansion of the image bit depth. Since the high-bit image has a higher dynamic range than the low-bit image, an inverse image processing module is used to perform image conversion on the high-bit standard color channel image with a higher dynamic range, so that the high-bit original image can have a high dynamic range. This method allows the image conversion to be performed directly by operating on the high-bit original image without the information of the original Raw image, so the difficulty of implementation is greatly reduced. At the same time, in the implementation process of some image processing, such as adjusting the effect and style of a photographic image, and the denoising and super-resolution tasks of a convolutional neural network based on a high-bit original image, the operation is more convenient and the image processing effect is better, thereby greatly improving the image quality and enhancing the image display effect. At the same time, the above-mentioned bit depth extension module and inverse image processing module are used to realize the conversion of low-bit standard color channel image to original high-bit image, which can realize end-to-end data processing. Because it is based on data-driven learning, the data conversion process is simpler and more effective, which improves the image conversion efficiency and does not introduce additional noise, thereby saving system overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is an internal structure diagram of a computer device in one embodiment;

[0038] Figure 2 is a schematic flow chart of an image conversion method in one embodiment;

[0039] Figure 3 is a flow chart of an image conversion method in another embodiment;

[0040] Figure 4 is a flowchart of an image conversion method in yet another embodiment;

[0041] Figure 5 is a flowchart of an image conversion method in yet another embodiment;

[0042] Figure 6 A schematic diagram of the network structure of a BDE module and a reverse ISP module in one embodiment;

[0043] Figure 7 FIG. 4 is a structural block diagram of an image conversion device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 1 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a bit depth extension module and an inverse image processing module. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image conversion method is implemented.

[0046] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0047] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0048] In one embodiment, Figure 2 As shown, an image conversion method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0049] S11. Use a bit depth extension module to perform bit depth extension processing on the low-bit standard color channel image to obtain a high-bit standard color channel image.

[0050] Specifically, the computer device may read a low-bit standard color channel image to be converted stored in a storage device or received from another device. The low-bit standard color channel image may be a low-bit sRGB image, for example, sRGB lowThe computer device can call the bit depth extension module to perform bit depth extension processing on the above-mentioned low-bit standard color channel image, so that the bit depth of the processed image changes from low to high, thereby obtaining a high-bit standard color channel image with a high bit depth. The high-bit standard color image can be a high-bit sRGB, for example, it can be recorded as sRGB high Optionally, the bit depth extension (BDE) module can be used to extend the bit depth of the image, extending the low bitmap to the high bit to obtain the corresponding high bitmap, for example, expanding the 4-bit map to 8 bits, 10 bits, 16 bits or other high bitmaps higher than 4 bits. Of course, the low bitmap can also be 2 bits, 8 bits, etc., and the corresponding high bitmap can be a high bitmap with a higher bit depth than the low bitmap. This embodiment does not specifically limit the specific bit depth.

[0051] S12, using an inverse image processing module to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0052] Specifically, the computer device uses a preset inverse image processing module, namely, an inverse ISP module, to perform image conversion on the above-mentioned high-bit standard color channel image to obtain a high-bit original image after format conversion. high The image is converted using an inverse ISP module to obtain a high-bit Raw image. Optionally, the inverse image processing module may be a module obtained by defining the inverse process of converting a high-bit original image into a high-bit standard color channel image. Optionally, it may be a module obtained by defining the inverse process of converting a high-bit Raw image into an sRGB image. Therefore, the inverse image processing module can realize the conversion from an sRGB image to a Raw image, for example, from Optionally, the inverse image processing module may also be a module including two network branches, including a branch for obtaining a color conversion matrix, and a branch for performing a nonlinear transformation, and the high-bit original image after the image conversion is obtained by fusion of the output data of these two branches.

[0053] In this embodiment, the computer device uses a bit depth extension module to perform bit depth extension processing on the low-bit standard color channel map to obtain a high-bit standard color channel map, thereby realizing the expansion of the image bit depth. Since the high-bit map has a higher dynamic range than the low-bit map, the inverse image processing module is used to perform image conversion on the high-bit standard color channel map with a higher dynamic range, so that the high-bit original map can have a high dynamic range. This method allows the image conversion to be performed directly by operating the high-bit original map without the information of the original Raw map, so the implementation difficulty is greatly reduced. At the same time, in the implementation process of some image processing, such as adjusting the effect and style of photographic images, and the denoising and super-resolution tasks of the convolutional neural network based on the high-bit original map, the operation is more convenient, the image processing effect is better, and the image quality is greatly improved, and the image display effect is improved. At the same time, the above-mentioned bit depth extension module and the inverse image processing module are used to realize the conversion of the low-bit standard color channel map to the original high-bit map, which can realize end-to-end data processing. Because it is based on the data-driven learning method, the data conversion process is simpler and more effective, the image conversion efficiency is improved, and no additional noise is introduced, thereby saving system overhead.

[0054] In one embodiment, the bit depth extension module includes a convolution submodule and an interpolation submodule; a possible implementation of the above step S11 can be as follows: Figure 3 As shown, including:

[0055] S111. Use the convolution submodule to perform convolution processing on the low-bit standard color channel image to obtain an intermediate low-bit color channel feature image.

[0056] Specifically, the bit depth expansion module may include a convolution submodule and an interpolation submodule. Among them, the convolution submodule may include multiple convolution layers and other network layers corresponding to the convolution layers. Optionally, the convolution submodule may be a residual jump structure without downsampling, and the number of convolution layers of the convolution submodule is not limited in this embodiment. The low-bit standard color channel map enters the convolution submodule to implement convolution processing, which may be a convolution operation for performing image feature extraction to obtain an intermediate low-bit color channel feature map.

[0057] S112. Use the target activation function layer to predict coefficients of the intermediate low-bit standard color channel image to obtain normalized interpolation coefficients.

[0058] Specifically, the computer device can use the target activation function layer to predict the coefficients of the above-mentioned intermediate low-bit standard color channel map, thereby obtaining the normalized interpolation coefficients. It should be noted that the activation function of the above-mentioned target activation function layer can be an activation function type that can output the normalized interpolation coefficients. When the above-mentioned intermediate low-bit standard color channel is input into the target activation function layer, the target activation function layer can predict the above-mentioned normalized interpolation coefficients.

[0059] S113, using the interpolation submodule to process the normalized interpolation coefficient, and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel image, to obtain the high-bit standard color channel image.

[0060] Specifically, the computer device can obtain the pixel value of each pixel point in the above low-bit standard color channel image, and select the maximum pixel value and the minimum pixel value therefrom, which can be recorded as l and max and l min The computer device can use an interpolation submodule based on the l max , l min The normalized interpolation coefficients are interpolated to achieve bit depth expansion and obtain a high-bit standard color channel image with a high bit depth.

[0061] Optionally, the activation function of the target activation function layer is a softmax function; the interpolation submodule includes l pre =α*l max +(1-α)*l min The relationship between pre is the pixel value of a pixel in the high-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, l max is the minimum pixel value in the pixel points of the low-bit standard color channel map, and α and (1-α) are normalized interpolation coefficients. It should be noted that the above softmax function can output normalized interpolation coefficients α and (1-α). Therefore, since the interpolation coefficients obtained by the softmax function are normalized, there will not be a large difference. Even if the bit depth extension module is not trained and uses randomly set network parameters, it can still ensure the output of high-precision high-bit standard color channel maps. At the same time, each pixel point of the above high-bit standard color channel map adopts the above relationship l pre =α*l max +(1-α)*l min Or the formula is obtained by a deformation, so that the operation of bit depth expansion is easy to implement, the efficiency of image conversion is improved and the resources of computer equipment are saved.

[0062] Here for high bitmap, that is, sRGB highand low-bitmap, i.e. sRGB low The principle of the conversion between the two is introduced in detail: the bit depth of the high-bitmap is h, the bit depth of the low-bitmap is l, and after normalization, the minimum resolution scale of the high-bitmap is 1 / (2 h -1), the minimum resolution scale of the low-level image is 1 / (2 l -1). For every 1 change in the low-bit image, the corresponding bit in the high-bit image is 2. h-l In the same interval, when the high-bit image is converted to the low-bit image, the transformation under the high-bit image is smoothed out, which can be regarded as the same low-bit value. Based on this principle, the pixel value of a pixel point on the low-bit image is l cur The maximum value of max and the minimum value l max as follows:

[0063] l max = l cur +2 (h-l) / (2 h -1)

[0064] l min = l cur -2 (h-l) / (2 h -1)

[0065] Then, using the formula l pre =α*l max +(1-α)*l min Get the pixel value of each point in the high-bit standard color channel image.

[0066] In this embodiment, the computer device performs convolution processing on the low-bit standard color channel map by using a convolution submodule to obtain an intermediate low-bit color channel feature map, and then uses a target activation function layer to predict the coefficients of the intermediate low-bit standard color channel map to obtain a normalized interpolation coefficient, and uses an interpolation submodule to process the normalized interpolation coefficient and the maximum pixel value and the minimum pixel value of each pixel in the low-bit standard color channel map to obtain a high-bit standard color channel map. Since the method uses a target activation function layer to predict the coefficients of the intermediate low-bit standard color channel map to obtain the normalized interpolation coefficient, the value of the normalized interpolation coefficient will not have too large a deviation, so the high-bit standard color channel map is determined based on the normalized interpolation coefficient, the maximum pixel value and the minimum pixel value of each pixel in the low-bit standard color channel map, and the error of the obtained high-bit standard color channel map is controllable, so the error is greatly reduced, and the image quality is greatly improved.

[0067] Optionally, on the basis of the above embodiments, the bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes multiple pairs of low-bit color training images and corresponding high-bit color training images. The above color training images include multiple pairs of low-bit color training images and corresponding high-bit color training images, and each pair of low-bit color training images and high-bit color training images are two images with different bit depths and the same source. Specifically, the computer device can input the above color training image set into the bit depth extension module and perform training, thereby updating the network parameters of the bit depth extension module, so that the updated bit depth extension module is more accurate, and then a more reasonable normalized interpolation coefficient can be obtained. Interpolation processing is performed based on the more reasonable normalized interpolation coefficient, further reducing the error of the obtained high-bit standard color channel map, thereby further improving the image quality.

[0068] Optionally, based on the above embodiments, a possible implementation of step S12 may be as follows: Figure 4 As shown, including:

[0069] S121, using the color space conversion submodule to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix.

[0070] It should be noted that the above-mentioned inverse image processing module may include two branches, namely, a branch of a color conversion submodule and a branch of a monotone nonlinear transformation submodule. Specifically, the computer device inputs the above-mentioned high-bit standard color channel map into a color space conversion submodule, and obtains a color space conversion matrix by learning the global. Optionally, the color space conversion submodule may be a model structure of progressive downsampling, and finally learns and outputs a 3*3 color space conversion matrix.

[0071] S122, using the monotone nonlinear transformation submodule to perform nonlinear processing on the high-bit standard color channel image to obtain a linearized linear feature image.

[0072] It should be noted that the above monotonic nonlinear transformation submodule includes a convolutional neural network model with monotonicity. The traditional convolutional neural network model regards the output of the model as the output of the final function, which can be recorded as y=f(x). The learning results of the traditional convolutional neural network are difficult to ensure the monotonicity of the model. In order to ensure the monotonicity of the model, based on the above analysis, the output of the model is regarded as the derivative of the function to be learned, then the objective function can be expressed as y=∫f(x)dx. As long as the derivative is a positive value, the original function, that is, the objective function, is monotonic.

[0073] Specifically, the computer device uses a monotone nonlinear transformation submodule to perform nonlinear processing on the high-bit standard color channel image. Due to the monotonicity of the objective function, the nonlinear change process is combined with prior knowledge, which enables the monotone nonlinear transformation submodule to converge quickly after very few training steps. Even if the network parameters in the monotone nonlinear transformation submodule are randomly initialized parameters, the information of the original image can still be well preserved.

[0074] Optionally, the monotone nonlinear transformation submodule is a module including a monotone function using a Clenshaw-Curtis quadrature algorithm. Based on the Clenshaw-Curtis quadrature algorithm, while realizing forward reasoning, the corresponding back-propagation function can also be derived, so it is easy to use and has richer application scenarios.

[0075] S123: Use the color space conversion matrix to transform the linear feature map to obtain the high-bit original map.

[0076] Specifically, the color space conversion matrix is ​​used to transform the linear feature map, which may be a multiplication of the two, so as to obtain a high-bit original map.

[0077] Optionally, Figure 6 Schematic diagram of the structure of a BDE module and an inverse ISP module in one embodiment.

[0078] In the present embodiment, the color space conversion submodule is used to perform color space conversion prediction based on the high-order standard color channel map to obtain a color space conversion matrix, and the monotone nonlinear transformation submodule is used to perform nonlinear processing on the high-order standard color channel map to obtain a linearized linear feature map, and the linear feature map and the color space conversion matrix are fused to obtain a high-order original map. In the present embodiment, due to the monotonicity of the monotone nonlinear transformation submodule, prior knowledge is combined in the nonlinear change process, so that the monotone nonlinear transformation submodule can be quickly converged after very few training steps, and has good robustness for different data sets. In addition, even if the network parameters in the monotone nonlinear transformation submodule are randomized network parameters, that is, without training, the model output can still well retain the information of the original image, so that the acquisition efficiency of the monotone nonlinear transformation submodule is higher and the model accuracy is higher, and the image with high output accuracy is also ensured.

[0079] In one embodiment, the inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes multiple pairs of high-bit color training images and corresponding high-bit original training images. The above-mentioned color training images include multiple pairs of low-bit color training images and corresponding high-bit color training images, and each pair of low-bit color training images and high-bit color training images are two images with different bit depths and the same source. Specifically, the computer device can input the above-mentioned conversion training image set into the inverse image processing module and perform training, thereby updating the network parameters of the inverse image processing module, so that the updated inverse image processing module is more accurate, and then a linear feature map with smaller error and more accuracy can be obtained, and a more accurate high-bit original map can be further obtained, further reducing the error and improving the image quality. In this embodiment, due to the monotonicity of the monotone nonlinear transformation submodule, the corresponding back propagation function can be derived, so it can be applied to the inverse conversion of the linear feature map into a high-bit standard color channel map, without the need to introduce other networks for processing, so it is easy to use and has richer application scenarios.

[0080] Optionally, based on the above embodiments, the method can also be Figure 5 As shown, including:

[0081] S13, performing a target processing operation on the high-level original image to obtain a target high-level original image.

[0082] S14, using the inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image.

[0083] S15. Perform a quantization operation based on bit depth compression on the target high-bit color channel map to obtain the target low-bit color channel map.

[0084] Specifically, the computer device can also perform a target processing operation on the above-mentioned high-bit original image, and the target processing operation may include but is not limited to operations such as denoising, performing super-resolution tasks, and adjusting the image style, so as to obtain a processed target high-bit original image. Then the computer device uses the above-mentioned inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image, that is, to perform an inverse nonlinear transformation on the high-bit original image after the target processing operation to obtain a target high-bit color channel image sRGB. Finally, the computer device can perform a quantization operation based on bit depth compression on the target high-bit color channel image to obtain a target low-bit color channel image, thereby realizing the target processing operation in the case of the high-bit original image, the target low-bit color channel image. The high-bit original image is processed using the above steps to obtain a target low-bit color channel image, and the accuracy of the above-mentioned image conversion process can be verified based on the target low-bit color channel image. Since the high-bit original image has a high dynamic range, and the value of the high-bit original image is linearly related to the light radiation intensity of the scene, performing a target processing operation in the format of the high-bit original image can greatly improve the image quality.

[0085] Optionally, the specific implementation process of step S14 in the above embodiment can be as follows: Since the monotonic nonlinear transformation submodule in the inverse image processing module is monotonic, the inverse image processing module can also be used to restore the original high-bit sRGB image from the reconstructed Raw image. Specifically, it can be:

[0086] First, define the following symbols: The low-bit standard color channel image is recorded as sRGB low ; The high-bit standard color channel image after the BDE module is recorded as sRGB high ; Raw represents the original high-level image finally predicted by the inverse ISP module, Branch CM Represents the color space conversion matrix output by the color space conversion submodule; Branch NL Represents the output of the monotone nonlinear transformation submodule, and sRGB is restored according to Raw high The steps are as follows:

[0087] First, Branch CM The inverse operation is called Branch CM -1 , then there is a relation Branch NL =Branch CM -1 Raw, due to Branch NL and sRGB high With a monotonic relationship, the problem can be converted into a known branch. NL Solving sRGB highThe problem can be solved by using the binary search method. During the solution process, the operation is iterated continuously and converged within the time of log(n) to complete the solution. Specifically, remember sRGB high The value of a point on input , Branch NL The corresponding value is y, which corresponds to the input x input The maximum value of the initialization may be x max =1, the minimum value is x min =0, the iteration steps are as follows:

[0088] A: xmid = (xmax + xmin) / 2;

[0089] B: Get x through the reverse ISP module mid The corresponding output is denoted as y mid ;

[0090] C: If y mid is greater than y, then let y mid =x max , otherwise, let y mid =x min ;

[0091] D: Repeat steps A to C until y mid -y satisfies the preset accuracy requirement, and the solution is completed. Optionally, the above accuracy requirement can be related to the bit depth. When the high bit depth is p, the accuracy requirement can be set to 1 / 2 p .

[0092] Optionally, when the bit depth of the low-bit standard color channel image in the above embodiment is 8 bits and the bit depth of the high-bit original image is 10 bits or 16 bits, the obtained image quality is higher.

[0093] The technical solution of the embodiment of the present application can also be evaluated using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural SIMilarity). The specific data are as follows:

[0094] The experimental results using the BDE module are compared with the data shown in Table 1 (ZP algorithm, Unet algorithm, this application (Our) and BitNet), where the value before " / " is the PSNR value and the value after is the SSIM value:

[0095] Table 1

[0096] ZP Unet Our BitNet 8->16, random parameters 52.96 / 0.9979 12.82 / 0.1652 52.60 / 0.9981 9.80 / -0.2424 8->16, after training 52.96 / 0.9997 58.29 / 0.9991 62.06 / 0.9995 58.42 / 0.9989 8->10, random parameters 54.77 / 0.9989 8.72 / 0.1670 54.19 / 0.9989 6.56 / -0.2325 8->10, after training 54.77 / 0.9989 58.45 / 0.9991 60.70 / 0.9993 56.85 / 0.9985 4->10, random parameters 28.74 / 0.8120 8.71 / 0.1665 28.28 / 0.8125 6.56 / -0.2329 4->10, after training 28.74 / 0.8120 41.49 / 0.9727 40.19 / 0.9690 38.56 / 0.9641

[0097] It can be seen from Table 1 that even if the same untrained BDE module is used, the PSNR value and SSIM value obtained are close to the traditional ZP algorithm (zero-breaking algorithm), and the effect is much better than the model of other algorithms. The data in Table 1 can show that when the bit depth is extended from 8 bits to 10 bits or 16 bits, the effect is better.

[0098] The inverse ISP module is used to evaluate the inherent data sets S7 and SID, and the data in Table 2 are obtained:

[0099] Table 2

[0100] Reverse ISP module BDE+Reverse ISP module S7 Datasets 24.73 / 0.9092 25.08 / 0.91 SID Dataset 52.46 / 0.9987 54.72 / 0.9990

[0101] It can be seen from Table 2 that the BDE module can further improve the image quality.

[0102] It should be understood that although Figure 2-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0103] In one embodiment, Figure 7 As shown, an image conversion device is provided, comprising:

[0104] The bit depth expansion module 100 is used to perform bit depth expansion processing on the low-bit standard color channel image to obtain a high-bit standard color channel image;

[0105] The inverse image processing module 200 is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0106] In one embodiment, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; the bit depth extension module 100 is specifically used to use the convolution submodule to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map; use the target activation function layer to perform coefficient prediction on the intermediate low-bit standard color channel map to obtain normalized interpolation coefficients; use the interpolation submodule to process the normalized interpolation coefficients and the maximum pixel value and minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

[0107] In one embodiment, the activation function of the target activation function layer is a softmax function; the interpolation submodule includes: pre =α*l max +(1-α)*l min The relationship between pre is the pixel value of a pixel in the high-bit standard color channel image, l min is the minimum pixel value among the pixels of the low-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, and α and (1-α) are normalized interpolation coefficients.

[0108] In one embodiment, the bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes a plurality of pairs of low-bit color training images and corresponding high-bit color training images.

[0109] In one embodiment, the inverse image processing module includes a color conversion submodule and a monotonic nonlinear transformation submodule; the inverse image processing module 200 is specifically used to use the color space conversion submodule to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix; use the monotonic nonlinear transformation submodule to perform nonlinear processing on the high-bit standard color channel map to obtain a linearized linear feature map; use the color space conversion matrix to transform the linear feature map to obtain the high-bit original map.

[0110] In one embodiment, the monotone nonlinear transformation submodule is a module including a monotone function using a Crenshaw-Curtis orthogonal algorithm.

[0111] In one embodiment, the inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes a plurality of pairs of high-bit color training images and corresponding high-bit original training images.

[0112] In one embodiment, the device also includes a restoration module, which is used to perform a target processing operation on the high-bit original image to obtain a target high-bit original image; use the inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image; perform a quantization operation based on bit depth compression on the target high-bit color channel image to obtain the target low-bit color channel image.

[0113] In one embodiment, the bit depth of the low-bit standard color channel image is 8 bits, and the bit depth of the high-bit original image is 10 bits or 16 bits.

[0114] The specific definition of the image conversion device can be found in the definition of the image conversion method above, which will not be repeated here. Each module in the above-mentioned image conversion device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0115] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0116] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0117] An inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0118] In one embodiment, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; when the processor executes the computer program, the following steps are also implemented:

[0119] The convolution submodule is used to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map;

[0120] Using the target activation function layer to predict the coefficients of the intermediate low-bit standard color channel image to obtain normalized interpolation coefficients;

[0121] The interpolation submodule is used to process the normalized interpolation coefficient and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

[0122] In one embodiment, the activation function of the target activation function layer is a softmax function; the interpolation submodule includes: pre =α*lmax +(1-α)*l min The relationship between

[0123] Among them, l pre is the pixel value of a pixel in the high-bit standard color channel image, l min is the minimum pixel value among the pixels of the low-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, and α and (1-α) are normalized interpolation coefficients.

[0124] In one embodiment, the bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes a plurality of pairs of low-bit color training images and corresponding high-bit color training images.

[0125] In one embodiment, the inverse image processing module includes a color conversion submodule and a monotonic nonlinear transformation submodule; when the processor executes the computer program, the following steps are also implemented:

[0126] The color space conversion submodule is used to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix;

[0127] The monotone nonlinear transformation submodule is used to perform nonlinear processing on the high-bit standard color channel image to obtain a linearized linear feature image;

[0128] The linear feature map is transformed using the color space conversion matrix to obtain the high-bit original map.

[0129] In one embodiment, the monotone nonlinear transformation submodule is a module including a monotone function using a Crenshaw-Curtis orthogonal algorithm.

[0130] In one embodiment, the inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes a plurality of pairs of high-bit color training images and corresponding high-bit original training images.

[0131] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0132] Performing a target processing operation on the high-level original image to obtain a target high-level original image;

[0133] Using the inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image;

[0134] A quantization operation based on bit depth compression is performed on the target high-bit color channel map to obtain the target low-bit color channel map.

[0135] In one embodiment, the bit depth of the low-bit standard color channel image is 8 bits, and the bit depth of the high-bit original image is 10 bits or 16 bits.

[0136] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0137] A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image;

[0138] An inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image.

[0139] In one embodiment, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; when the computer program is executed by the processor, the following steps are also implemented:

[0140] The convolution submodule is used to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map;

[0141] Using the target activation function layer to predict the coefficients of the intermediate low-bit standard color channel image to obtain normalized interpolation coefficients;

[0142] The interpolation submodule is used to process the normalized interpolation coefficient and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

[0143] In one embodiment, the activation function of the target activation function layer is a softmax function; the interpolation submodule includes: pre =α*l max +(1-α)*l min The relationship between pre is the pixel value of a pixel in the high-bit standard color channel image, l min is the minimum pixel value among the pixels of the low-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, and α and (1-α) are normalized interpolation coefficients.

[0144] In one embodiment, the bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes a plurality of pairs of low-bit color training images and corresponding high-bit color training images.

[0145] In one embodiment, the inverse image processing module includes a color conversion submodule and a monotone nonlinear transformation submodule; when the computer program is executed by the processor, the following steps are also implemented:

[0146] The color space conversion submodule is used to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix;

[0147] The monotone nonlinear transformation submodule is used to perform nonlinear processing on the high-bit standard color channel image to obtain a linearized linear feature image;

[0148] The linear feature map is transformed using the color space conversion matrix to obtain the high-bit original map.

[0149] In one embodiment, the monotone nonlinear transformation submodule is a module including a monotone function using a Crenshaw-Curtis orthogonal algorithm.

[0150] In one embodiment, the inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes a plurality of pairs of high-bit color training images and corresponding high-bit original training images.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] Performing a target processing operation on the high-level original image to obtain a target high-level original image;

[0153] Using the inverse image processing module to perform an inverse nonlinear transformation on the target high-bit original image to obtain a target high-bit color channel image;

[0154] A quantization operation based on bit depth compression is performed on the target high-bit color channel map to obtain the target low-bit color channel map.

[0155] In one embodiment, the bit depth of the low-bit standard color channel image is 8 bits, and the bit depth of the high-bit original image is 10 bits or 16 bits.

[0156] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0157] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image conversion method, characterized in that: The method comprises: A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image; Using an inverse image processing module to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image; Among them, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; the bit depth extension module is used to perform bit depth expansion processing on the low-bit standard color channel map to obtain a high-bit standard color channel map, including: using the convolution submodule to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map; using the target activation function layer to perform coefficient prediction on the intermediate low-bit standard color channel map to obtain a normalized interpolation coefficient; using the interpolation submodule to process the normalized interpolation coefficient, and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

2. The method according to claim 1, characterized in that The activation function of the target activation function layer is a softmax function; the interpolation submodule includes l pre =α*l max +(1-α)*l min The relationship between Among them, l pre is the pixel value of a pixel in the high-bit standard color channel image, l min is the minimum pixel value among the pixels of the low-bit standard color channel image, l max is the maximum pixel value among the pixels of the low-bit standard color channel image, and α and (1-α) are normalized interpolation coefficients.

3. The method according to claim 1, characterized in that The bit depth extension module is a model obtained by training with a color training image set; wherein the color training image set includes a plurality of pairs of low-bit color training images and corresponding high-bit color training images.

4. The method according to any one of claims 1 to 3, characterized in that: The inverse image processing module includes a color conversion submodule and a monotone nonlinear transformation submodule; the inverse image processing module is used to perform image conversion on the high-bit standard color channel image to obtain a high-bit original image, including: Using a color space conversion submodule to perform color space conversion prediction based on the high-bit standard color channel map to obtain a color space conversion matrix; The monotone nonlinear transformation submodule is used to perform nonlinear processing on the high-bit standard color channel image to obtain a linearized linear feature image; The linear feature map is transformed using the color space conversion matrix to obtain the high-bit original map.

5. The method according to claim 4, characterized in that The monotone nonlinear transformation submodule is a module including a monotone function using the Crenshaw-Curtis orthogonal algorithm.

6. The method according to claim 4, characterized in that The inverse image processing module is a model obtained by training with a conversion training image set; wherein the conversion training image set includes a plurality of pairs of high-bit color training images and corresponding high-bit original training images.

7. The method according to claim 4, characterized in that The bit depth of the low-bit standard color channel image is 8 bits, and the bit depth of the high-bit original image is 10 bits or 16 bits.

8. An image conversion device, characterized in that: The device comprises: A bit depth expansion module is used to perform bit depth expansion processing on a low-bit standard color channel image to obtain a high-bit standard color channel image; An inverse image processing module, used for performing image conversion on the high-bit standard color channel image to obtain a high-bit original image; Among them, the bit depth extension module includes a convolution submodule, a target activation function layer and an interpolation submodule; the bit depth extension processing of the low-bit standard color channel map to obtain a high-bit standard color channel map includes: using the convolution submodule to perform convolution processing on the low-bit standard color channel map to obtain an intermediate low-bit color channel feature map; using the target activation function layer to perform coefficient prediction on the intermediate low-bit standard color channel map to obtain a normalized interpolation coefficient; using the interpolation submodule to process the normalized interpolation coefficient, and the maximum pixel value and the minimum pixel value of each pixel point in the low-bit standard color channel map to obtain the high-bit standard color channel map.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images