Contrast adjustment method based on curve mapping, electronic equipment and imaging system

Abstract

The invention provides a contrast adjustment method based on curve mapping, electronic equipment and an imaging system. The contrast adjustment method based on curve mapping comprises the following steps: acquiring a low dynamic range image obtained by pre-compressing a to-be-compressed high dynamic range image and pre-compression brightness information of the to-be-compressed high dynamic range image; splicing the pre-compressed brightness information into the to-be-compressed high dynamic range image to obtain an artificial feature layer; inputting the artificial feature layer into a preset neural network model to obtain a color gradation mapping curve; the preset neural network model is used for performing multi-scale feature extraction on the artificial feature layer and generating the color gradation mapping curve based on the multi-scale features; and performing contrast adjustment on the low dynamic range image based on a color gradation mapping curve. According to the method, the display effect of the low-dynamic-range image obtained through compression can be improved under the condition that a small amount of calculation and power consumption are increased, and the real-time performance of image output is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, provides a contrast adjustment method based on curve mapping, an electronic device, and an imaging system. Background Art

[0002] In application scenarios such as video surveillance that include CMOS (Complementary Metal-Oxide-Semiconductor) image sensors, it is often necessary to capture high dynamic range images that can simultaneously display details in dark areas of the image while preserving content in bright areas.

[0003] However, some devices, such as mobile phones, TVs, and tablets, cannot directly display HDR (High Dynamic Range Imaging) images. Furthermore, in some scenarios, image content detection, recognition, and tracking are required. Existing detection, recognition, and tracking algorithms typically process low-dynamic-range images. Therefore, there is a need to compress high-dynamic-range images into low-dynamic-range images.

[0004] Existing algorithms for compressing high-dynamic-range images, on the one hand, need to further improve the display quality of compressed low-dynamic-range images. On the other hand, some existing compression algorithms generate a large amount of computational effort when improving the display quality of low-dynamic-range images. For example, using a Unet (U-shaped Network) requires predicting pixel brightness information, calculating residual connections, and computing pixel tone mapping. Using such networks to improve image display quality can significantly increase computational effort, increase power consumption, and increase computation time, impacting the real-time performance of compressed image output. Summary of the Invention

[0005] In view of this, the present application aims to provide a contrast adjustment method, electronic device and imaging system based on curve mapping, so as to increase the display effect of the compressed low dynamic range image while reducing the compression calculation amount and power consumption, and improve the real-time performance of the image output.

[0006] First, an embodiment of the present application provides a contrast adjustment method based on curve mapping, including: obtaining a low dynamic range image obtained by pre-compressing a high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed; splicing the pre-compressed brightness information into the high dynamic range image to be compressed to obtain an artificial feature layer; inputting the artificial feature layer into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model; the preset neural network model is used to perform multi-scale feature extraction on the artificial feature layer and generate the color scale mapping curve based on the multi-scale features, wherein the multi-scale features include features of different scales under preset dimensions; and performing contrast adjustment on the low dynamic range image based on the color scale mapping curve.

[0007] After pre-compression, the brightness information of a high dynamic range image may have a problem of poor display effect. In an embodiment of the present application, after obtaining the pre-compressed brightness information of the high dynamic range image to be compressed, the pre-compressed brightness information is spliced ​​with the high dynamic range image to be compressed, and then the artificial feature layer obtained by splicing is input into a neural network model to perform multi-scale feature extraction on the spliced ​​artificial feature layer through the neural network model. The neural network model can learn the artificial feature layer features to generate a tone mapping curve that is more suitable for the high dynamic range image. Based on the tone mapping curve, the contrast of the low dynamic range image corresponding to the high dynamic range is further adjusted, which can improve the display effect of the compressed low dynamic range image. In addition, the neural network model in the present application only needs to calculate the color level mapping curve, and does not need to perform pixel-by-pixel prediction of the image, calculate residual connections, or calculate the tone mapping of each pixel in the image as in existing compression methods. This can effectively reduce power consumption, improve computing efficiency, and thereby improve the compression efficiency of high dynamic range images. Therefore, the use of this neural network model can improve the display effect of low dynamic range images, improve the compression efficiency of high dynamic range images, and improve the real-time performance of image output with less increase in computational complexity and power consumption.

[0008] In one embodiment, obtaining a low dynamic range image obtained by pre-compressing a high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed includes: obtaining image brightness information of the high dynamic range image to be compressed; pre-compressing the image brightness information to obtain the pre-compressed brightness information and the low dynamic range image.

[0009] In one embodiment, the pre-compressing the image brightness information includes: using a preset logarithmic function to pre-compress the dynamic range of the image brightness information to obtain the pre-compressed brightness information and the low dynamic range image.

[0010] In the embodiment of the present application, a logarithmic function is used for pre-compression, which is computationally simple and generates less power consumption, thereby helping to improve pre-compression efficiency and overall compression efficiency of high dynamic range images.

[0011] In one embodiment, the high dynamic range image to be compressed is an RGB (Red, Green, Blue) format image; obtaining the image brightness information of the high dynamic range image to be compressed includes: converting the RGB format image into a YUV (a color encoding method) format image; extracting a brightness component Y from the YUV format image; the brightness component Y represents the image brightness information; or, converting the RGB format image into an HSV (Hue, Saturation, Value, hexagonal pyramid model) format image; extracting luminance information V from the HSV format image; the luminance information V represents the image brightness information.

[0012] In the embodiment of the present application, two different formats of images, HSV format image and YUV format image, can be used for image compression, which helps to improve the versatility and compatibility of the method provided by the present application and expand the scope of application.

[0013] In one embodiment, the preset neural network model includes multiple encoders with the same structure and multiple fully connected layers with the same structure; the output end of each encoder is connected to the input end of one of the fully connected layers; the artificial feature layer is input into the preset neural network model to obtain the color scale mapping curve output by the preset neural network model, including: adjusting the artificial feature layer into sub-feature layers of different scales; the number of the sub-feature layers matches the number of the encoders; the sub-feature layers of different scales are input into different encoders respectively to obtain the local curve features output by the fully connected layers to which each encoder is connected; the color scale mapping curve is obtained based on the splicing of different local curve features under the preset dimension.

[0014] In the embodiment of the present application, the neural network model only has an encoder and a fully connected layer. Compared with the Unet network that includes both an encoder and a decoder, it effectively simplifies the structure of the neural network model, reduces the computational complexity of the neural network model, reduces the power consumption of the neural network model, and improves the image compression efficiency and output real-time performance of the neural network model.

[0015] In one embodiment, the preset neural network model also includes a self-attention layer, which is connected to the output ends of different fully connected layers; the color scale mapping curve is obtained by splicing different local curve features under the preset dimension, including: splicing the different local curve features under the preset dimension to obtain initial multi-scale features; inputting the initial multi-scale features into the self-attention layer for dimensional compression to obtain target multi-scale features under the preset dimension; the preset dimension matches the dimension of the high dynamic range image to be compressed; and mapping the target multi-scale features into a color scale mapping curve based on a preset mapping function.

[0016] In an embodiment of the present application, after different local curve features are spliced ​​together, the dimension of the initial multi-scale feature is the sum of the dimensions of the different local curve features, which cannot be directly used for image compression. Therefore, the initial multi-scale feature is dimensionally compressed and then mapped into a color scale mapping curve so that the color scale mapping curve can compress the high dynamic range image to be compressed.

[0017] In one embodiment, adjusting the artificial feature layer into sub-feature layers of different scales includes: adjusting the artificial feature layer into preset sub-features of different scales based on a preset bilinear interpolation method.

[0018] In the embodiment of the present application, bilinear interpolation is a commonly used and simple image scaling algorithm. Using this algorithm helps to reduce computational complexity, thereby reducing the computational power consumption of the neural network model, improving computational efficiency, and thus improving the efficiency of image compression.

[0019] In one embodiment, the tone mapping curve includes the coordinates of the pixel, the correspondence between the brightness value after contrast adjustment and the low dynamic range brightness value, and contrast adjustment of the low dynamic range image is performed based on the tone mapping curve, including: compressing the low dynamic range image to the preset dimension; for each pixel in the low dynamic range image at the preset dimension, determining the low dynamic range brightness value of the pixel in the tone mapping curve based on the coordinates of the pixel and the image brightness information, to obtain the low dynamic range image after contrast adjustment.

[0020] In an embodiment of the present application, compression can be performed through a color scale mapping curve in a manner similar to a table lookup, that is, the contrast-adjusted brightness value corresponding to the pixel is searched in the color scale mapping curve based on the pixel's coordinates and the low dynamic range brightness value. This method is simple to implement and helps reduce the power consumption and computing time required for image compression, thereby improving image compression efficiency.

[0021] Based on the same inventive concept, an embodiment of the present application provides a curve-based image compression device, including: an acquisition module, used to acquire a low dynamic range image obtained by pre-compressing a high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed; a splicing module, used to splice the pre-compressed brightness information into the high dynamic range image to be compressed to obtain an artificial feature layer; a mapping module, used to input the artificial feature layer into a preset neural network model to obtain a color scale mapping curve; the preset neural network model is used to perform multi-scale feature extraction on the artificial feature layer and generate the color scale mapping curve based on the multi-scale features, wherein the multi-scale features include features of different scales under preset dimensions; a contrast adjustment module, used to adjust the contrast of the low dynamic range image based on the color scale mapping curve.

[0022] In a third aspect, an embodiment of the present application provides an electronic device comprising: a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the contrast adjustment method based on curve mapping as described in any one of the first aspects.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is run on a computer, the computer executes the contrast adjustment method based on curve mapping as described in any one of the first aspects.

[0024] In a fifth aspect, an embodiment of the present application provides an imaging system, comprising: an image acquisition device; and the electronic device as described in the third aspect, communicatively connected to the image acquisition device. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those skilled in the art, other relevant drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic flow chart of a contrast adjustment method based on curve mapping provided in one embodiment of the present application; Figure 2 A processing diagram of a neural network model provided in one embodiment of the present application; Figure 3 A schematic diagram of an image compression effect provided by an embodiment of the present application; Figure 4A schematic diagram of a contrast adjustment device based on curve mapping provided in one embodiment of the present application; Figure 5 A schematic diagram of an electronic device provided in accordance with an embodiment of the present application.

[0027] Icon: contrast adjustment device 200 based on curve mapping; acquisition module 210; stitching module 220; mapping module 230; contrast adjustment module 240; electronic device 300; processor 310; memory 320. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0029] See also Figure 1 , Figure 1 This is a flow chart of a contrast adjustment method based on curve mapping provided in one embodiment of the present application. The contrast adjustment method based on curve mapping includes: S110 , obtaining a low dynamic range image obtained by pre-compressing the high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed.

[0030] In the embodiment of the present application, the pre-compressed brightness information refers to the brightness information obtained after compressing the image brightness information of the high dynamic range image to be compressed, that is, the pre-compressed brightness information is the brightness information of the low dynamic range image.

[0031] In an embodiment of the present application, obtaining a low dynamic range image obtained by pre-compressing a high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed may include: obtaining image brightness information of the high dynamic range image to be compressed, and then pre-compressing the image brightness information to obtain pre-compressed brightness information and a low dynamic range image.

[0032] In an embodiment of the present application, when a high dynamic range image is compressed to obtain a low dynamic range image, what is compressed is mainly the image brightness information of the high dynamic range image to be compressed. Therefore, the image brightness information can be directly pre-compressed to obtain pre-compressed brightness information and a low dynamic range image.

[0033] In the embodiment of the present application, the high dynamic range image to be compressed may be an RGB image, or an image in other formats, which is converted into an RGB image through an existing format conversion method.

[0034] Different formats of high dynamic range images may correspond to different image brightness information. For example, for a high dynamic range image in HSV format, the image brightness information can be represented by the brightness information V. For another example, for a high dynamic range image in YUV format, the image brightness information can be represented by the brightness component Y.

[0035] Therefore, if the high dynamic range image to be compressed is an RGB format image, in one embodiment of the present application, the process of obtaining image brightness information may include: converting the RGB format image into a YUV format image, and then extracting the brightness component Y from the YUV format image.

[0036] In another embodiment of the present application, the process of obtaining image brightness information may include: converting the RGB format image into an HSV format image, and then extracting brightness information V from the HSV format image.

[0037] The process of image conversion in the above embodiment can be referred to the prior art and will not be elaborated here.

[0038] The process of pre-compressing image brightness information provided in an embodiment of the present application may include: using a preset logarithmic function to pre-compress the dynamic range of the image brightness information to obtain pre-compressed brightness information.

[0039] In this embodiment, the preset logarithmic function can be expressed as:

[0040] Wherein, N is the pre-compressed brightness information, Luma is the uncompressed image brightness information, and Luma is a matrix, Max represents the maximum value, and log2 represents logarithmic calculation.

[0041] In some other embodiments of the present application, other existing methods may also be used to pre-compress image brightness information, which is not limited here.

[0042] S120 , splicing the pre-compressed brightness information into the high dynamic range image to be compressed to obtain an artificial feature layer.

[0043] In an embodiment of the present application, the high dynamic range image to be compressed in RGB format does not include brightness information, and compressing the high dynamic range image to a low dynamic range image includes compressing the brightness. Therefore, when compressing the high dynamic range image, the brightness information can be spliced ​​into the high dynamic range image so that the high dynamic range image to be compressed includes brightness information.

[0044] In the embodiments of the present application, the spliced ​​luminance information is pre-compressed luminance information, i.e., luminance information that has been compressed once. In the aforementioned context, compressing image luminance information is equivalent to compressing the luminance information of a high dynamic range image to a low dynamic range. However, the pre-compression method used for the global image is the same. The resulting pre-compressed luminance information fails to reflect the image characteristics of each part of the image and fails to utilize the voice information in the image, resulting in a poor display quality for the compressed low dynamic range image.

[0045] Therefore, in an embodiment of the present application, the pre-compressed luminance information is spliced ​​into the high dynamic range image to be compressed, so as to be adjusted in combination with the local features of the high dynamic range image, thereby obtaining a tone mapping curve for high dynamic range image compression that can provide a better display effect.

[0046] In an embodiment of the present application, the high dynamic range image to be compressed in RGB format includes features of three channels: R, G and B. After splicing the pre-compressed luminance information, it will include features of four channels: R, G, B and pre-compressed luminance information.

[0047] In an embodiment of the present application, the luminance information is stitched into the high dynamic range image, and the stitching can be performed based on a preset pre-compressed luminance information stitching relationship, wherein the pre-compressed luminance information stitching relationship can be expressed as:

[0048] Among them, Feature represents the artificial feature layer, and the symbol [] represents the splicing algorithm. Represents the characteristics of the high dynamic range image to be compressed, including the characteristics of the three channels R, G, and B. N represents the pre-compressed brightness information, Luma represents the uncompressed image brightness information, and Luma is a matrix.

[0049] S130: Input the artificial feature layer into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model.

[0050] See also Figure 2 , Figure 2 A processing diagram of a neural network model provided in one embodiment of the present application.

[0051] In the embodiment of the present application, the preset neural network model may include multiple encoders with the same structure and multiple fully connected layers with the same structure. Figure 2 For example, the neural network model in one embodiment includes 3 encoders, and each encoder is connected to a fully connected layer.

[0052] In an embodiment of the present application, the artificial feature layer is input into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model, which can include: adjusting the artificial feature layer into sub-feature layers of different scales; inputting the sub-feature layers of different scales into different encoders, respectively, and obtaining the local curve features output by the fully connected layer connected to each encoder; and obtaining the color scale mapping curve based on the splicing of different local curve features under a preset dimension.

[0053] In this embodiment of the present application, the sub-feature layer is an artificial feature layer that has been rescaled (or image scaled). The rescaling ratio can be 1, meaning that the scale of the sub-feature layer is the same as that of the artificial feature layer. Furthermore, the number of sub-feature layers matches the number of encoders. For example, if the number of encoders is 3, the number of sub-feature layers is also 3.

[0054] For example, Figure 2 As shown, in one embodiment of the present application, the scale of the high dynamic range image to be compressed and the pre-compressed luminance information are both 1024×1024, and the number of both is 3. Therefore, both can be represented as 3×1024×1024. After the two are spliced, the scale of the obtained artificial feature layer is 1024×1024, and the number of artificial feature layers is 6. Therefore, the artificial feature layer before rescaling can be represented as 6×1024×1024. At the same time, the 6×1024×1024 artificial feature layer can also be used as a sub-feature layer. The artificial feature layer can then be resized into sub-feature layers of 512×512 and 256×256 scales, respectively, to obtain a 6×512×512 sub-feature layer and a 6×256×256 sub-feature layer. Thus, a total of three sub-feature layers of 1024×1024, 512×512, and 256×256 scales can be obtained, and the number of each sub-feature layer is 6.

[0055] The neural network model in this application only needs to calculate the color scale mapping curve, without the need to perform pixel-by-pixel prediction, calculate residual connections, or calculate the tone mapping of each pixel in the image, as in existing compression methods. This can effectively reduce power consumption, improve computational efficiency, and thus improve the compression efficiency of high dynamic range images. Therefore, using this neural network model can improve the display effect of low dynamic range images, improve the compression efficiency of high dynamic range images, and improve the real-time performance of image output while increasing the amount of computation and power consumption.

[0056] In one embodiment of the present application, the artificial feature layer can be adjusted to preset sub-features of different scales based on a preset Bilinear (bilinear interpolation) algorithm. The principle and use of the bilinear interpolation method can be referred to the existing technology and will not be elaborated here. The bilinear interpolation method is a commonly used and simple image scaling algorithm. Using this algorithm helps to reduce computational complexity, thereby reducing the computational power consumption of the neural network model, improving the computational efficiency of the neural network model, and thus improving the efficiency of image compression.

[0057] In other embodiments of the present application, other existing image scaling algorithms may also be used, which will not be elaborated here.

[0058] In the embodiment of the present application, the artificial feature layer is adjusted to sub-feature layers of different scales and respectively input into the neural network model to learn the features of high dynamic range images at different scales through different encoders and fully connected layers, thereby obtaining the local curve features corresponding to the sub-feature layers of different scales. Among them, the feature dimensions of the local curve features output by different fully connected layers are the same, for example, Figure 2 As shown, they are all 64×16×16, where 64 is the channel dimension of the feature and 16x16 is the spatial dimension of the image. When adjusting the scale, the scale to be adjusted is also determined based on the spatial dimension of the image. The scale to be adjusted must be larger than the spatial dimension of the image. For example, 16x16 is 2 4 ×2 4 , 256×256 is 2 8 ×2 8 , 512×512 is 2 9 ×2 9 , and so on.

[0059] Different from the Unet network, the neural network model provided in this application includes an encoder and a fully connected layer, but does not include a decoder. Figure 2 As shown in the figure, the sub-feature layer passes through the encoder and fully connected layers before being output without decoding. Compared to the UNet network, which includes both encoders and decoders, this neural network model, consisting of only encoders and fully connected layers, effectively simplifies its structure, reduces its computational complexity, lowers its power consumption, and improves its image compression efficiency and real-time output.

[0060] The result output by the fully connected layer is a local curve feature, which has a better compression effect on high dynamic range images of their respective scales. Therefore, in an embodiment of the present application, different local curve features can be spliced ​​to obtain a color scale mapping curve, which can be corrected based on the local curve features corresponding to different scales, so that the color scale mapping curve has a better adjustment effect on different parts of the image, thereby presenting a better display effect.

[0061] In one embodiment of the present application, the preset neural network model further includes a self-attention layer connected to the outputs of different fully connected layers. A color scale mapping curve is obtained by concatenating different local curve features, including: concatenating the different local curve features at a preset dimension to obtain an initial multi-scale feature; inputting the initial multi-scale feature into the self-attention layer for dimensionality compression to obtain a target multi-scale feature at the preset dimension; and mapping the target multi-scale feature into a color scale mapping curve based on a preset mapping function.

[0062] In this embodiment of the present application, the initial multi-scale features output by the fully connected layer are used to concatenate different local curve features to obtain the initial multi-scale features. The self-attention layer is configured with a self-attention mechanism. The self-attention layer performs dimensionality reduction on the initial multi-scale features based on the self-attention mechanism to obtain the target multi-scale features of a preset dimension. In the embodiment of the present application, the fully connected layer splices different local curve features, which can be expressed as:

[0063] Among them, Multi-Scale Feature represents the tone mapping curve, and F1, F2, and F3 represent local feature curves corresponding to different scales. When more scales are adjusted, F4, F5, etc. can also be included. When only two scales are adjusted, F3 may not be included.

[0064] Among them, stitching under the preset dimension means adjusting the local curve features to the same preset dimension, for example, Figure 2 As shown, all local curve features are adjusted to 64×16×16, where 64 is the channel dimension of the feature and 16×16 is the spatial dimension of the feature.

[0065] like Figure 2 As shown in the figure, the dimension of the initial multi-scale feature obtained by splicing different local curve features is the sum of the features of different local feature curves. For example, the dimension of the local curve feature is 64x16x16, and the dimension of the initial multi-scale feature is 192x16x16. The initial multi-scale feature of this dimension cannot be directly used to compress the high dynamic range image to be compressed. Therefore, the initial multi-scale feature needs to be dimensional compressed to compress the spliced ​​feature to the preset dimension.

[0066] In an embodiment of the present application, the initial multi-scale features can be input into the self-attention layer for dimensional compression to obtain target multi-scale features of a preset dimension. The preset dimension matches the dimension of the high dynamic range image to be compressed, for example, compressed to the same dimension as the local curve feature, such as compressed to 64x16x16, so as to obtain the target multi-scale features of a preset dimension. Figure 2For example, the dimension of the local curve feature is 64x16x16. The dimension of the initial multi-scale feature obtained by concatenating three local curve features is 192x16x16. The dimension of the initial multi-scale feature is compressed, and the dimension of the target multi-scale feature obtained by compression is 64x16x16.

[0067] The self-attention mechanism has the function of dimensionality compression. For details, please refer to the existing technology and will not be explained here.

[0068] It should be noted that the encoder, fully connected layer, self-attention layer (and its configured self-attention mechanism) in the neural network model can refer to the existing technology. The encoder, fully connected layer, and self-attention layer in the neural network model provided in this application do not require additional deformation. Therefore, the specific implementation of the neural network model will not be expanded here.

[0069] After obtaining the target multi-scale features, the target multi-scale features can be mapped into a color scale mapping curve using a preset mapping function, thereby performing image compression using the color scale mapping curve. There are many types of mapping functions, which are not limited here. In one embodiment of the present application, a Sigmoid function can be used for mapping.

[0070] S140: Adjust the contrast of the low dynamic range image based on the tone mapping curve.

[0071] The tone mapping curve includes the correspondence between pixel coordinates, low dynamic range brightness values, and brightness values ​​after contrast adjustment. Pixel coordinates refer to the coordinates of pixels in the low dynamic range image or high dynamic range image. When a high dynamic range image is compressed to obtain a low dynamic range image, the pixel coordinates are the same. Low dynamic range brightness values ​​refer to the brightness values ​​of each pixel in the low dynamic range image before contrast adjustment, and contrast-adjusted brightness values ​​refer to the brightness values ​​of the pixels after contrast adjustment.

[0072] Based on the tone mapping curve, the contrast of the pre-compressed low dynamic range can be further adjusted to improve the display effect of the low dynamic range image.

[0073] For example, in one embodiment of the present application, image compression of a high dynamic range image to be compressed based on a color scale mapping curve may include: compressing a low dynamic range image to the preset dimension; for each pixel of the high dynamic range image to be compressed, determining a low dynamic range brightness value of the pixel in the color scale mapping curve based on the coordinates of the pixel and the image brightness information, to obtain a low dynamic range image.

[0074] In the embodiments of this application, pixel brightness adjustment should be on the same dimension. Therefore, if the low dynamic range image is not on the preset dimension, it needs to be compressed to the preset dimension. The compression method can use the aforementioned self-attention layer, which is based on the self-attention mechanism of the self-attention layer for compression, which will not be elaborated here.

[0075] In this embodiment of the present application, the coordinates of a pixel can be used as an index value to obtain the low dynamic range brightness value of the pixel in the low dynamic range image. The low dynamic range brightness value is then used to look up the table in the color scale mapping curve, and the corresponding low contrast adjusted brightness value of the pixel is determined using the color scale mapping curve. The contrast adjustment of the low dynamic range image is completed until the corresponding contrast-adjusted brightness values ​​are determined for all pixels.

[0076] This method can determine the corresponding brightness value after contrast adjustment by looking up the color level mapping curve. Compared with other calculation methods, when the color level mapping curve has been obtained, the table lookup requires less calculation and lower power consumption, which helps to improve the efficiency of image contrast adjustment.

[0077] See also Figure 3 , Figure 3 This is a schematic diagram of the image compression effect provided by an embodiment of the present application, with a high dynamic range image on the left and a low dynamic range image on the right. Figure 3 As shown in the figure, the high dynamic range image cannot see the details clearly due to its high brightness. However, through the method provided in this application, the low dynamic range image on the right can be obtained. The low dynamic range image can effectively display the details and reduce the impact of excessive brightness.

[0078] Based on the same inventive concept, the present application embodiment also provides an image compression based on curve mapping, see Figure 4 , Figure 4 Schematic diagram of a contrast adjustment device based on curve mapping provided by an embodiment of the present application. The contrast adjustment device based on curve mapping 200 includes: an acquisition module 210 , a splicing module 220 , a mapping module 230 and a contrast adjustment module 240 .

[0079] The acquisition module 210 is configured to acquire a low dynamic range image obtained by pre-compressing the high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed.

[0080] The splicing module 220 is configured to splice the pre-compressed brightness information into the high dynamic range image to be compressed to obtain an artificial feature layer.

[0081] The mapping module 230 is used to input the artificial feature layer into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model.

[0082] The contrast adjustment module 240 is configured to perform contrast adjustment on the low dynamic range image based on the tone mapping curve.

[0083] In one embodiment, the acquisition module 210 is configured to acquire image brightness information of the high dynamic range image to be compressed; and pre-compress the image brightness information to obtain the pre-compressed brightness information and the low dynamic range image.

[0084] In one embodiment, the acquisition module 210 is configured to pre-compress the dynamic range of the image brightness information using a preset logarithmic function to obtain the pre-compressed brightness information and the low dynamic range image.

[0085] In one embodiment, the high dynamic range image to be compressed is an RGB format image, and the acquisition module 210 is used to convert the RGB format image into a YUV format image; extract the brightness component Y from the YUV format image; the brightness component Y represents the brightness information of the image; or, convert the RGB format image into an HSV format image; extract the luminance information V from the HSV format image; the luminance information V represents the brightness information of the image.

[0086] In one embodiment, the preset neural network model includes multiple encoders with the same structure and multiple fully connected layers with the same structure; the output of each encoder is connected to the input of a fully connected layer. A mapping module 230 is configured to adjust the artificial feature layer into sub-feature layers of different scales; the number of sub-feature layers matches the number of encoders; input the sub-feature layers of different scales into different encoders to obtain local curve features output by the fully connected layers to which each encoder is connected; and obtain the color scale mapping curve based on the concatenation of the different local curve features in the preset dimension.

[0087] In one embodiment, the preset neural network model also includes a self-attention layer, which is connected to the output ends of different fully connected layers. The mapping module 230 is used to splice the different local curve features under the preset dimension to obtain initial multi-scale features; input the initial multi-scale features into the self-attention layer for dimensional compression to obtain target multi-scale features under the preset dimension; the preset dimension matches the dimension of the high dynamic range image to be compressed; and map the target multi-scale features into a color scale mapping curve based on a preset mapping function.

[0088] In one embodiment, the mapping module 230 is configured to adjust the artificial feature layer into preset sub-features of different scales based on a preset bilinear interpolation method.

[0089] In one embodiment, the tone scale mapping curve includes the tone scale mapping curve including the coordinates of the pixel, the low dynamic range brightness value and the brightness value after contrast adjustment, and the contrast adjustment is performed on the low dynamic range image based on the tone scale mapping curve. The contrast adjustment module 240 is used to compress the low dynamic range image to the preset dimension; for each pixel in the low dynamic range image under the preset dimension, the low dynamic range brightness value of the pixel is determined in the tone scale mapping curve based on the coordinates and image brightness information of the pixel, so as to obtain the low dynamic range image after contrast adjustment.

[0090] The function implemented by the contrast adjustment device 200 based on curve mapping is similar to the aforementioned contrast adjustment method based on curve mapping, and will not be described in detail here.

[0091] The embodiment of the present application further provides an electronic device 300, which can be used as the execution subject of the above-mentioned contrast adjustment method based on curve mapping. Figure 5 , Figure 5 Schematic diagram of an electronic device according to an embodiment of the present application. The electronic device 300 includes a processor 310 and a memory 320 communicatively connected to the processor 310 .

[0092] The memory 320 stores instructions that can be executed by the processor 310 . The instructions are executed by the processor 310 so that the processor 310 can perform the contrast adjustment method based on curve mapping in the aforementioned embodiment.

[0093] The processor 310 and the memory 320 may be connected via a communication bus, or via some communication modules, such as a wireless communication module, a Bluetooth communication module, a 4G / 5G communication module, etc.

[0094] The processor 310 may be an integrated circuit chip with signal processing capabilities, such as a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), an AI (Artificial Intelligence), an NPU (Neural Network Processing Unit), an ISP (Image Signal Processor), a DPU (Display Processing Unit), a VPU (Video Processing Unit), or a DSP (Digital Signal Processor), which is a data processing core. The processor 310 may also be a processor chip used in scenarios such as large-scale data computing. The above is merely an example and should not be construed as limiting the present application.

[0095] The memory 320 may include, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0096] It is understandable that the electronic device 300 may also include more common modules required by itself, which are not introduced one by one in the embodiments of the present application.

[0097] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the method provided in the above embodiment is executed.

[0098] The storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as an SSD).

[0099] In the embodiments provided herein, it should be understood that the disclosed methods and devices may also be implemented in other ways. The device embodiments described above are merely illustrative. The functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0100] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk, and other media that can store program codes.

[0101] Based on the same inventive concept, an embodiment of the present application further provides an imaging system, which includes an image acquisition device and the electronic device provided in the aforementioned embodiment. The image acquisition device can be various cameras or various devices with cameras.

[0102] In an embodiment of the present application, the imaging system may be a monitoring system, the image acquisition device may be a monitoring camera, and the electronic device may be a mobile phone, a computer, a server, etc.

[0103] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0104] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A contrast adjustment method based on curve mapping, characterized in that: include: Obtaining a low dynamic range image obtained by pre-compressing the high dynamic range image to be compressed and pre-compressed brightness information of the high dynamic range image to be compressed; splicing the pre-compressed brightness information into the high dynamic range image to be compressed to obtain an artificial feature layer; Inputting the artificial feature layer into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model; the preset neural network model is used to extract multi-scale features from the artificial feature layer and generate the color scale mapping curve based on the multi-scale features, wherein the multi-scale features include features of different scales under a preset dimension; Contrast adjustment is performed on the low dynamic range image based on the tone mapping curve.

2. The contrast adjustment method based on curve mapping according to claim 1, characterized in that: The obtaining of the low dynamic range image obtained by pre-compressing the high dynamic range image to be compressed and the pre-compressed brightness information of the high dynamic range image to be compressed includes: Acquiring image brightness information of the high dynamic range image to be compressed; The image brightness information is pre-compressed to obtain the pre-compressed brightness information and the low dynamic range image.

3. The contrast adjustment method based on curve mapping according to claim 2, characterized in that: The pre-compressing the image brightness information includes: The dynamic range of the image brightness information is pre-compressed using a preset logarithmic function to obtain the pre-compressed brightness information and the low dynamic range image.

4. The contrast adjustment method based on curve mapping according to claim 2, characterized in that: The high dynamic range image to be compressed is an RGB format image; The acquiring of image brightness information of the high dynamic range image to be compressed includes: Converting the RGB format image into a YUV format image; extracting a brightness component Y from the YUV format image; wherein the brightness component Y represents brightness information of the image; Alternatively, the RGB format image is converted into an HSV format image; and luminance information V is extracted from the HSV format image; the luminance information V represents the brightness information of the image.

5. The contrast adjustment method based on curve mapping according to claim 1, characterized in that: The preset neural network model includes a plurality of encoders with the same structure and a plurality of fully connected layers with the same structure; the output end of each encoder is connected to the input end of one of the fully connected layers; Inputting the artificial feature layer into a preset neural network model to obtain a color scale mapping curve output by the preset neural network model includes: Adjusting the artificial feature layer into sub-feature layers of different scales respectively; the number of the sub-feature layers matches the number of the encoders; Inputting the sub-feature layers of different scales into different encoders respectively, and obtaining the local curve features output by the fully connected layer connected to each encoder; The color scale mapping curve is obtained based on the splicing of different local curve features under the preset dimension.

6. The contrast adjustment method based on curve mapping according to claim 5, characterized in that: The preset neural network model further includes a self-attention layer, which is connected to the output ends of different fully connected layers; the color scale mapping curve is obtained by splicing different local curve features under the preset dimension, including: Splicing the different local curve features under the preset dimension to obtain initial multi-scale features; Inputting the initial multi-scale features into the self-attention layer for dimensionality compression to obtain target multi-scale features at the preset dimension; the preset dimension matches the dimension of the high dynamic range image to be compressed; The target multi-scale feature is mapped into a color scale mapping curve based on a preset mapping function.

7. The contrast adjustment method based on curve mapping according to claim 5, characterized in that: Adjusting the artificial feature layer into sub-feature layers of different scales includes: adjusting the artificial feature layer into preset sub-features of different scales based on a preset bilinear interpolation method.

8. The contrast adjustment method based on curve mapping according to any one of claims 1 to 7, characterized in that: The color scale mapping curve includes coordinates of pixels, low dynamic range brightness values, and brightness values ​​after contrast adjustment, and contrast adjustment is performed on the low dynamic range image based on the color scale mapping curve, including: compressing the low dynamic range image to the preset dimension; For each pixel in the low dynamic range image under the preset dimension, a low dynamic range brightness value of the pixel is determined in the color scale mapping curve based on the coordinates of the pixel and the image brightness information, thereby obtaining a low dynamic range image after contrast adjustment.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the contrast adjustment method based on curve mapping according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The readable storage medium stores a computer program, and when the computer program is run on a computer, the computer is caused to execute the contrast adjustment method based on curve mapping according to any one of claims 1 to 8.

11. An imaging system, characterized in that: include: Image acquisition equipment; The electronic device according to claim 9, being communicatively connected to the image acquisition device.

Images