Image tone mapping method and device, electronic equipment, and storage medium

Abstract

The embodiment of the present disclosure relates to an image tone mapping method and device, electronic equipment and storage medium, and relates to the technical field of images. The image tone mapping method comprises the following steps: obtaining a to-be-processed image, and dividing the to-be-processed image to obtain a plurality of image blocks; performing instance segmentation on the to-be-processed image to obtain the category of each pixel, and determining the mapping parameters corresponding to each category; combining the mapping parameters corresponding to the category of all pixels in each image block to determine the tone mapping curve of each image block in the to-be-processed image; and performing tone mapping on each image block based on the tone mapping curve of each image block to obtain a target image. The technical solution in the embodiment of the present disclosure can improve the accuracy of local tone mapping.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image technology, and in particular, to an image tone mapping method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] In the image processing process, image tone mapping processing is a common way to improve image quality.

[0003] In the related art, tone mapping can be performed on the entire image based on one parameter, or local tone mapping. Among them, global tone mapping has certain limitations, and the accuracy of local tone mapping is low, which makes the tone mapping effect poor and reduces the image quality.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present disclosure is to provide an image tone mapping method and device, an electronic device, and a storage medium, thereby at least partially overcoming the problem of poor tone mapping image quality caused by the limitations and defects of the related art.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to a first aspect of the present disclosure, an image tone mapping method is provided, comprising: obtaining a to-be-processed image, and dividing the to-be-processed image to obtain a plurality of image blocks; performing instance segmentation on the to-be-processed image to obtain the category of each pixel, and determining the mapping parameter corresponding to each category; determining the tone mapping curve of each image block in the to-be-processed image in combination with the mapping parameter corresponding to the category of all pixels in each image block; and performing tone mapping on each image block based on the tone mapping curve of each image block to obtain a target image.

[0008] According to a second aspect of the present disclosure, an image tone mapping device is provided, comprising: an image dividing module configured to obtain a to-be-processed image, and divide the to-be-processed image to obtain a plurality of image blocks; a mapping parameter obtaining module configured to perform instance segmentation on the to-be-processed image to obtain the category of each pixel, and determine the mapping parameter corresponding to each category; a mapping curve determining module configured to determine the tone mapping curve of each image block in the to-be-processed image in combination with the mapping parameter corresponding to the category of all pixels in each image block; and a tone mapping module configured to perform tone mapping on each image block based on the tone mapping curve of each image block to obtain a target image.

[0009] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the image tone mapping method of the first aspect and possible implementation manners thereof by executing the executable instructions.

[0010] According to a fourth aspect of the present disclosure, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the image tone mapping method of the first aspect and possible implementation manners thereof.

[0011] In the technical solutions provided in the embodiments of the present disclosure, on the one hand, the tone mapping parameter of each image block can be determined in combination with the mapping parameter of the category of each pixel obtained by performing instance segmentation on the to-be-processed image, and then the tone mapping curve of each image block is determined. Since each image block has a corresponding tone mapping curve, local tone mapping can be performed independently, avoiding the limitation that only global tone mapping can be performed in the related art, and flexibility and pertinence are increased. On the other hand, the tone mapping curve of each image block can be guided according to the category of the pixels of each image block, the tone mapping curve of each image block can be determined from multiple dimensions such as pixel category and image block, the accuracy and authenticity of the tone mapping curve of each image block are improved, corresponding tone mapping can be implemented for pixels of different categories, the accuracy of local tone mapping is improved, and the image quality is improved.

[0012] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0014] Figure 1 A schematic diagram showing an application scenario of the image tone mapping method to which the embodiments of the present disclosure can be applied is shown.

[0015] Figure 2 A schematic diagram of an image tone mapping method according to an embodiment of the present disclosure is shown.

[0016] Figure 3 A schematic diagram of pixel recombination in an embodiment of the present disclosure is shown.

[0017] Figure 4A schematic diagram of a UNET network in an embodiment of the present disclosure is shown.

[0018] Figure 5 A flowchart of a process of acquiring a category of a pixel in an embodiment of the present disclosure is shown.

[0019] Figure 6 A schematic diagram of determining a tone mapping parameter of an image block in an embodiment of the present disclosure is shown.

[0020] Figure 7 A flowchart of a process of tone mapping in an embodiment of the present disclosure is shown.

[0021] Figure 8 A schematic diagram of an interpolation process in an embodiment of the present disclosure is shown.

[0022] Figure 9 A flowchart of a process of image processing in an embodiment of the present disclosure is shown.

[0023] Figure 10 A flowchart of a process of tone mapping an image in an embodiment of the present disclosure is shown.

[0024] Figure 11 A block diagram of an image tone mapping apparatus in an embodiment of the present disclosure is shown.

[0025] Figure 12 A block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0027] In addition, the accompanying drawings are only schematic and are non-limiting. Identical components have been given the same reference numerals in the various drawings and have not been repeatedly described. Some embodiments of the apparatuses shown in the drawings are functional entities that may not necessarily have a corresponding physical or logical entity in an implementation, and may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] In the related art, tone mapping can be performed by Reinhard tone mapping. Specifically, the mean value of the current pixel corresponding to the Log domain conversion is counted, and the pixel value in the image is mapped to another value according to the mean value, so as to map one reference brightness to neutral gray. In addition, the value of the image can be converted in the Log domain, and each pixel value is further tone mapped according to the maximum value and the minimum value of the entire image input.

[0029] In the above manner, the global tone mapping manner determined by the global mean value has the problem of poor local effect. When processing by local tone mapping, there is a problem of poor accuracy, which leads to poor image quality.

[0030] To solve the technical problems in the related art, an image tone mapping method is provided in the embodiments of the present disclosure, which can be applied to tone mapping processing of images in a photographing process to realize an application scenario of color enhancement. Figure 1 A schematic diagram of a system architecture to which the image tone mapping method and device of the embodiments of the present disclosure can be applied is shown.

[0031] As shown in Figure 1 The terminal 101 can be a smart device with image processing function, for example, a smart phone, a computer, a tablet computer, a smart speaker, a smart watch, a vehicle-mounted device, a wearable device, a monitoring device, etc. The terminal can include a camera, which can be of any type as long as it can perform photographing. The number of cameras can be at least one, for example, one, four, etc. as long as it can perform photographing. The image to be processed can be an image obtained by photographing, or each frame of image in a video obtained by photographing.

[0032] In the embodiment of the present disclosure, the terminal 101 can include a memory 102 and a processor 103. The memory is used to store images, and the processor is used to process images, such as tone mapping processing and the like. The memory 102 can store a to-be-processed image 104. The terminal 101 obtains the to-be-processed image 104 from the memory 102 and sends it to the processor 103, and the to-be-processed image is divided into a plurality of image blocks in the processor 103; the to-be-processed image is instance segmented to obtain the category of each pixel, and the mapping parameters corresponding to each category are determined; the mapping parameters corresponding to all the categories of the pixels in each image block are combined to determine the tone mapping curve of each image block in the to-be-processed image; and each image block is tone mapped based on the tone mapping curve of each image block, thereby generating a target image 105 after tone mapping.

[0033] It should be noted that the image tone mapping method provided by the embodiment of the present disclosure can be executed by the terminal 101. The image tone mapping method can also be provided in the terminal.

[0034] Figure 2 The image tone mapping method in the embodiment of the present disclosure is schematically shown in the flowchart, which specifically includes the following steps:

[0035] In step S210, a to-be-processed image is obtained, and the to-be-processed image is divided into a plurality of image blocks.

[0036] In step S220, the to-be-processed image is instance segmented to obtain the category of each pixel, and the mapping parameters corresponding to each category are determined.

[0037] In step S230, the mapping parameters corresponding to all the categories of the pixels in each image block are combined to determine the tone mapping curve of each image block in the to-be-processed image.

[0038] In step S240, each image block is tone mapped based on the tone mapping curve of each image block to obtain a target image.

[0039] In the embodiment of the present disclosure, the to-be-processed image can be any type of image, and the to-be-processed image can be divided into a plurality of image blocks. The sizes of the plurality of image blocks can be the same or different, and the size and number of the image blocks are specifically determined according to actual needs.

[0040] In addition, the image to be processed can be subjected to instance segmentation to obtain the category of each pixel in the image to be processed, i.e., which category each pixel belongs to. The categories can include, but are not limited to, sky, human, grass greenery, light, black body, and other types. The categories can be determined according to actual needs. After obtaining the category of each pixel, the mapping parameter of each category can be determined. The mapping parameter is used to represent the mapping degree of each category. It should be noted that the steps of dividing the image into blocks to obtain a plurality of image blocks and performing instance segmentation on the image to be processed to obtain the category of each pixel can be interchanged, i.e., the image division can be performed first or the instance segmentation can be performed first, which is not specifically limited here.

[0041] Further, the categories of all pixels contained in each image block can be counted, the tone mapping parameter of the image block is collectively determined according to the mapping parameters of all categories contained in each image block, and then the tone mapping curve of each image block is determined in combination with the tone mapping parameter of the target image block. On this basis, each image block can be subjected to local tone mapping based on the tone mapping curve of each image block, so as to realize tone mapping of the entire image to be processed, obtain the target pixel value of each pixel point in each image block, and thus obtain the target image after tone mapping corresponding to the image to be processed.

[0042] Next, the specific steps of the image tone mapping method in the embodiment of the present disclosure will be described in detail with reference to Figure 2

[0043] In step S210, an image to be processed is obtained, and the image to be processed is divided into a plurality of image blocks.

[0044] In the embodiment of the present disclosure, the image to be processed can be an image obtained by photographing an object to be photographed by a camera module of a terminal, or each frame of image in a photographed video. The terminal can be any one of a smartphone, a digital camera, a smart watch, a wearable device, a vehicle-mounted device, or a camera of a monitoring device, as long as it can take a photograph of an object to be photographed and can realize image processing. Herein, a smartphone is taken as an example for description. The camera module can include at least one camera, such as any one or a combination of a main camera, a telephoto camera, a wide-angle camera, and a macro camera. The image to be processed can be various types of images, such as a dynamic image or a static image, etc.

[0045] ​The to-be-processed image can be an RGB image, i.e., an RGB three-channel image. Each pixel point of the RGB image is composed of three colors of RGB. When the terminal is in a shooting mode, a shooting image obtained by a camera module can be a RAW image. The RAW image is original image data information collected by the camera module. In the embodiments of the present disclosure, the shooting image obtained by the terminal can be converted to obtain the to-be-processed image. For example, the shooting image in RAW format can be converted to the to-be-processed image in RGB format by using a general conversion algorithm to improve the convenience of subsequent processing.

[0046] The to-be-processed image can also be a Bayer format image, as shown in Figure 3 When the to-be-processed image is a Bayer format image, the to-be-processed image can be pixel-recombined and arranged to reduce the input bandwidth of the network. The specific steps of pixel-recombination and arrangement can include: recombining and arranging the to-be-processed image according to the color channel. For example, each color channel is recombined into an image to obtain a recombined image, which is not limited here.

[0047] After obtaining the to-be-processed image, the to-be-processed image can be divided into multiple image blocks. The image block can be a part of the to-be-processed image, and the multiple image blocks do not overlap. The size of each image block can be the same or different, and the number of image blocks can be determined according to the division method. In order to facilitate processing, the size of each image block is taken as an example for illustration here. The division method can be determined according to actual needs.

[0048] In the embodiments of the present disclosure, the number of image blocks can be determined according to multiple reference attribute parameters. The multiple reference attribute parameters can include, but are not limited to, any one or combination of the following: field of view feature, image resolution, light source state, algorithm performance, and evaluation parameter. The field of view feature can be the field of view and the light source feature, the light source state can be the number and distribution of light sources, the algorithm performance can be the amount of calculation, and the evaluation parameter can be the tone mapping accuracy. Based on this, the number of image blocks obtained by image division can be determined according to actual needs based on the reference attribute parameters. For example, the to-be-processed image can be divided into MxN image blocks.

[0049] Next, continuing to refer to Figure 2 In step S220, the to-be-processed image is instance segmented to obtain the category of each pixel, and the mapping parameters corresponding to each category are determined.

[0050] In this embodiment of the disclosure, instance segmentation refers to separating the foreground and background of the image to be processed, achieving pixel-level object separation. Instance segmentation is performed on each object in the image to be processed, and each pixel can be assigned a category. This category can be used to describe which object each pixel in the image to be processed belongs to, such as whether each pixel belongs to the sky, a person, or a light source, etc. The category can be determined according to actual needs, and may include, but is not limited to: sky, person, grass and vegetation, light source, black body, and other types.

[0051] In some embodiments, an instance segmentation model can be used to segment the image to be processed into instances to obtain the category of each pixel. The instance segmentation model can be any type of model capable of instance segmentation, such as Mask-RCNN instance segmentation network or PANet instance segmentation network. Here, we take the UNET network as an example. UNET network refers to a U-shaped network.

[0052] Figure 4 The diagram illustrates the structure of the UNET network. (See reference...) Figure 4 As shown, the UNET network may include a first part and a second part. The first part may be a downsampling module consisting of two 3×3 convolutional layers and one 2×2 max-pooling layer repeated multiple times, which can be used to implement feature extraction. The second part is an upsampling module consisting of one upsampling convolutional layer, feature concatenation (copy and crop), and two 3×3 convolutional layers repeated multiple times, which can be used to implement feature fusion. The last layer of the second part transforms the number of channels into the desired number of categories through a 1×1 convolution. In this embodiment, the first part may perform multiple downsampling operations, and the second part may correspondingly perform multiple upsampling processes. For example, four downsampling operations and four upsampling operations may be performed.

[0053] Based on the above network structure, the specific process for obtaining the category of each pixel can be found in [reference needed]. Figure 5 As shown, the main steps include:

[0054] In step S510, the image to be processed is subjected to multiple feature extractions through a multi-layer downsampling network in the instance segmentation model to obtain the feature vector of each layer of the downsampling network;

[0055] In step S520, a corresponding reference feature vector is obtained based on the feature vector of each network layer. The reference feature vector and the feature vector are then fused through a multi-layer upsampling network to obtain the category of each pixel.

[0056] In the embodiments of the present disclosure, the up-sampling module of the first part can include a multi-layer network, each layer of the network including two 3x3 convolution layers and a max-pooling layer, so the multi-layer network can be considered as a multi-layer down-sampling network. When the first part performs feature extraction, since each layer of the down-sampling network includes two 3x3 convolution layers and a max-pooling layer, the convolution operation can be performed on the to-be-processed image according to each layer of the down-sampling network in sequence to obtain a convolution result; the convolution result is down-sampled to obtain a down-sampled result, so as to determine the down-sampled result as a feature vector of each layer of the down-sampling network. That is, the feature extraction is performed through the convolution layer and the max-pooling layer in the first layer of the down-sampling network in sequence, and the feature extraction is performed according to the convolution layer and the max-pooling layer in the second layer of the down-sampling network, until the feature extraction is performed through the convolution layer and the max-pooling layer in all layers of the down-sampling network. After each down-sampling, the number of channels is doubled.

[0057] The up-sampling module of the second part can include a multi-layer network, each layer of the network including one up-sampling convolution layer, feature splicing and cropping, and two 3x3 convolution layers, so the multi-layer network of the second part can be considered as a multi-layer up-sampling network. When the second part performs processing, the feature vector of each layer of the down-sampling network of the first part is up-sampled by each layer of the up-sampling network of the second part to obtain an up-sampled result. When the feature is spliced, the same size feature vector is first cut from the feature vector output by the same layer of the first part as the reference feature vector of each layer of the up-sampling network of the second part, and the reference feature vector is fused with the up-sampled result of the second part to obtain the fusion feature of each layer of the up-sampling network. The fusion here can be channel splicing. The convolution operation is further performed on the fusion feature of each layer of the up-sampling network to obtain the category of each pixel. After each up-sampling convolution, the height and width are doubled, and the number of channels is halved, which is used for merging and splicing with the shallow layer feature of the first part.

[0058] On the basis of Figure 4 , the to-be-processed image can be input into the image segmentation model, and the convolution layer and the max-pooling layer in the image segmentation model are used to repeatedly perform feature extraction on the to-be-processed image to obtain a feature vector of the to-be-processed image. Further, the up-sampling convolution layer in the image segmentation model is used for up-sampling processing to obtain an up-sampled result, the up-sampled result is combined and spliced with the feature vector of the same layer, and the convolution operation is performed, and the above steps are repeatedly executed to obtain a target feature vector; the convolution operation is performed on the target feature vector to obtain the category of each pixel.

[0059] For example, the image to be processed is input into the instance segmentation model, a convolution operation is performed on the image to be processed to obtain a first feature vector, and a down-sampling is performed on the first feature vector to obtain a first down-sampled result; a convolution operation is performed on the first down-sampled result to obtain a second feature vector, and a down-sampling is performed on the second feature vector to obtain a second down-sampled result; the above steps are repeated until the convolution operation and the down-sampling of all layers of the network are performed, for example, a fifth down-sampled result is obtained, and the down-sampled result of each layer of the down-sampling network is taken as the feature vector of the layer. After obtaining the feature vector of each layer of the down-sampling network, the second part of the feature fusion is performed based on the feature vector of the last layer. Specifically, the feature vector is up-sampled to obtain an up-sampled result, the up-sampled result is fused with the reference feature vector of the same layer to obtain a fused feature, and a convolution operation is performed on the fused feature. The above steps are repeatedly performed until the class of each pixel is obtained. The feature vector of the same layer itself can be taken as the reference feature vector, or the feature vector of the same layer is cropped to have the same size as the feature vector to obtain the reference feature vector.

[0060] In each down-sampling, the number of channels becomes twice that of the previous layer of the network; in each up-sampling, the number of channels becomes half that of the previous layer of the network. For example, as shown in Figure 4 , the number of channels changes from 64 to 1024 after 4 times of down-sampling; the number of channels changes from 1024 to 64 after 4 times of up-sampling.

[0061] In the embodiments of the present disclosure, the UNET network is used to obtain the class of each pixel, the output of the feature extraction part is fused in the up-sampling part, the multi-scale features are fused together, and the features extracted by the shallow network and the features extracted by the deep network can be fused to improve the comprehensiveness and accuracy of the features.

[0062] After obtaining the class of each pixel, the corresponding mapping parameter can be determined for each class. The mapping parameter can be used to represent the degree of tone mapping corresponding to each class. The mapping parameters corresponding to different classes can be different, and are specifically determined according to actual requirements. For example, the mapping parameter can be determined according to the color brightness or other attribute parameters of each class. For example, the classes obtained by segmentation, i.e., sky, people, grass green plants, light, black body and other types of parameters, are respectively set … .

[0063] Next, still referring to Figure 2 , in step S230, the tone mapping curve of each image block in the image to be processed is determined in combination with the mapping parameters corresponding to the classes of all pixels in the image block.

[0064] In the embodiments of the present disclosure, one image block can contain multiple pixels, and all the pixels in one image block can belong to the same category or different categories. Based on this, the tone mapping curve of each image block in the image to be processed can be determined according to the mapping parameters of the categories of all the pixels contained in each image block. The tone mapping curve can be a mapping function, which is used to represent the mapping relationship between the input value and the output value. After determining the tone mapping parameters of each image block, the tone mapping curve of each image block can be further generated according to the tone mapping parameters.

[0065] In some embodiments, if the categories of the pixels contained in the image block are different, different ways are used to determine the tone mapping parameters of the image block. For example, if all the pixels in the image block include one category, the tone mapping parameters are determined according to the mapping parameters of the category; if all the pixels in the image block include multiple categories, the mapping parameters of the multiple categories are fused to determine the tone mapping parameters. Specifically, if there is only one category in a certain image block, the mapping parameters of the category are directly determined as the tone mapping parameters of the image block. If there are multiple categories in a certain image block, the proportion of each category can be determined, and the tone mapping parameters of each image block are determined by performing addition operation according to the proportion of each category. The proportion of each category can be determined according to the product of the number of pixels of each category contained in each image block and the mapping parameters of each category, and the ratio of the product to the total number of pixels corresponding to the image block. The specific way of determining the tone mapping parameters of the image block can be shown in formula (1):

[0066] Formula (1)

[0067] wherein, is the number of pixels corresponding to the category, is the mapping parameter of each category, is the updated tone mapping parameter of the image block, N is the total number of pixels of the image block, and m is the number of categories contained in each image block.

[0068] In addition, a weight can be set for each category, and the weights of different categories can be the same or different. Further, the proportions of the categories contained in the image block can be added according to the weights to obtain the tone mapping parameters of each image block.

[0069] Figure 6 FIG. 1 schematically shows a flowchart of determining the tone mapping parameters of the image block. Referring to FIG. 1, Figure 6As shown in the figure, the image to be processed 600 is divided into multiple image blocks, the image block 601 contains one class, for example, a person, and the tone mapping parameter of the image block is determined according to the mapping parameter corresponding to the class person. The image block 602 contains two classes, for example, class 1 and class 2, class 1 can be a person, and class 2 can be grass green plants. Then, the proportion 1 of class 1 is determined according to the product of the mapping parameter corresponding to class 1 and the number of pixels of class 1, and the ratio of the product to the total number of pixels of the image block 602, and the proportion 2 of class 2 is determined according to the product of the mapping parameter corresponding to class 2 and the number of pixels of class 2, and the ratio of the product to the total number of pixels of the image block 502, and the tone mapping parameter of the image block 602 is determined according to the sum of the proportion 1 and the proportion 2. The tone mapping parameter of the entire image block is determined by the mapping parameter of the class of all pixels contained in each image block, which can improve the accuracy and comprehensiveness of the tone mapping parameter of the image block.

[0070] After obtaining the tone mapping parameter of each image block, a corresponding tone mapping curve can be generated for each image block, and the tone mapping curves of each image block can be the same or different, which is determined according to the specific tone mapping parameter obtained.

[0071] In determining the tone mapping curve, the instance segmentation result and the tone mapping parameter of the image block can be combined to determine. Specifically, the mean of each image block and the maximum value in the image to be processed are gamma mapped according to the tone mapping parameter of the target image block to generate the tone mapping curve of each image block.

[0072] The target image block can be a first image block and a second image block, the first image block can be a dark area, and the second image block can be a bright area. The dark area and the bright area can be determined according to the brightness value, for example, in a night scene with a light source, the image block area near the light source has a larger brightness (a larger gray value) and can be a bright area, and the image block area far from the light source has a smaller brightness (a smaller gray value) and can be a dark area. Specifically, the ratio of the mean of each image block to the maximum value in the image to be processed before mapping can be calculated to obtain the ratio of each image block, and the tone mapping curve of each image block can be obtained based on the ratio of each image block and the tone mapping parameter of the target image block. The fusion based on the ratio of each image block and the tone mapping parameter of the target image block is specifically a logical operation of the ratio of each image block and the debugging parameter, and a multiplication operation of the logical operation result and the tone mapping parameter of the target image block, so as to obtain the tone mapping curve of each image block. The logical operation can be an addition operation or a multiplication operation, etc., which is not limited here.

[0073] In the embodiments of the present disclosure, the tone mapping parameters of each image block are determined by combining the mapping parameters corresponding to the categories of all pixels in the image block, so that the tone mapping curve of each image block is determined according to the tone mapping parameters of the target image block, the mean value of each image block and the maximum value in the image to be processed. Since the tone mapping curve of each image block can be generated by combining the categories of semantic segmentation and the bright area and the dark area, the accuracy and pertinence of the tone mapping curve can be improved.

[0074] Reference Figure 2 As shown in FIG. 2, in step S240, each image block is tone mapped based on the tone mapping curve of each image block to obtain a target image.

[0075] In the embodiments of the present disclosure, each image block in the image to be processed can be tone mapped based on the tone mapping curve of each image block, so as to realize tone mapping of the entire image to be processed to obtain a target image. Tone mapping refers to compressing the dynamic range of an image to below the dynamic range of an output device, so that a high dynamic range image can adapt to a low dynamic range output device, that is, the process of mapping the pixel value (luminance value) of an image from a high dynamic range to a low dynamic range. Tone mapping is also a kind of nonlinear mapping, which can reduce the luminance of the pixel points in the bright area and then improve the luminance of the pixel points in the dark area. Tone mapping can be performed on luminance values, or can be performed in combination with color information, or can be performed on pixel values. In the embodiments of the present disclosure, mapping is performed on pixel values as an example.

[0076] In tone mapping, the pixel value of each pixel point in the image to be processed can be mapped to obtain a target pixel value by normalization and mapping. In addition, since tone mapping can be used to adjust the luminance value of each pixel point, the corresponding luminance value can be obtained based on the pixel value of each pixel point, and the target luminance value can be obtained by mapping the luminance value.

[0077] Figure 7 The flowchart of performing tone mapping is shown in FIG. 2, and reference is made to Figure 7 As shown in FIG. 2, the method mainly includes the following steps:

[0078] In step S710, the pixel value of each pixel point in each image block of the image to be processed is normalized and mapped to obtain the normalized pixel value of the pixel point.

[0079] In this step, the pixel point refers to each pixel point in each image block. The pixel value of each pixel point in the to-be-processed image can be normalized to obtain a normalized pixel value. The pixel value can be a gray value of the pixel point. The normalization manner can be any type of normalization manner, for example, can be determined according to the ratio of the pixel value of each pixel point to the maximum value in the to-be-processed image. In addition, the maximum value, the minimum value and the mean value in the to-be-processed image can also be combined to perform normalization. The maximum value, the minimum value and the mean value can be the maximum value, the minimum value and the mean value of the gray value. Illustratively, the pixel value of each pixel point can be converted in the log domain, and the pixel value of each pixel point can be normalized to obtain a normalized pixel value by combining the maximum value L max , the minimum value L min and the mean value La in the to-be-processed image, which is specifically shown in formula (2):

[0080] Formula (2)

[0081] The mean value can be calculated by formula (3):

[0082] Formula (3)

[0083] In step S720, the normalized pixel value is gamma mapped based on the tone mapping curve of the image block to which the pixel point belongs, to obtain a mapped pixel value of the pixel point.

[0084] In this step, the normalized pixel value can be tone mapped and mapped to a target value, thereby obtaining the mapped pixel value of the pixel point. The tone mapping can be realized by gamma mapping or histogram equalization. Here, gamma mapping is taken as an example for illustration.

[0085] Illustratively, the normalized pixel value can be processed by gamma mapping. Specifically, the normalized pixel value of the pixel point can be gamma mapped according to the tone mapping curve of the image block to which the pixel point belongs. For example, when gamma is equal to 1, gamma mapping is not needed. When the gamma parameter is less than 1, the image will become dark; when the gamma parameter is greater than 1, the image will become bright.

[0086] The normalized pixel value after gamma mapping is further expanded and mapped to a target value, so as to map the pixel value of the pixel point to the mapped pixel value. The target value can be the maximum value of the expected output after mapping. The process of obtaining the mapped pixel value is specifically shown in formula (4):

[0087] Formula (4)

[0088] ​​wherein p is a debugging parameter, is a maximum value of the input pixel before mapping, is a maximum value of the expected output after mapping, is a mean value of each image block. is a tone mapping parameter of the first image block, is a tone mapping parameter of the second image block.

[0089] By normalizing the pixel value of the pixel point, and performing gamma mapping on the normalized pixel value according to the tone mapping curve of the image block, and further mapping to the maximum value of the expected output, the accuracy of the mapped pixel value can be improved. In addition, the tone mapping parameter of the first image block and the tone mapping parameter of the second image block are used as two gamma correction coefficients, and the two gamma correction coefficients are used to control the dark area and the bright area of the image respectively, and then the gamma mapping is realized for each image block, which can flexibly realize the control of the brightness and the dark area, and increase the application range and the accuracy.

[0090] In some embodiments, the class of each pixel obtained by the instance segmentation result can also be applied to other local tone mapping methods. The other local tone mapping method can be a CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm. In the CLAHE algorithm, the enhancement amplitude of the local contrast can be limited by limiting the height of the local histogram of the image. In the CLAHE algorithm, the image is first divided into different image block regions and it is ensured that the image block regions do not overlap with each other, and then histogram equalization is performed for each image block region in the image; wherein a threshold for limiting the contrast is set for the height of the histogram of each image block region, the part exceeding the threshold can be cropped and evenly distributed to the gray values of 0-255, and finally the equalization operation is performed based on the reconstructed histogram.

[0091] When CLAHE is used for local tone mapping, a global histogram height preset threshold is preset for the image to be processed, and the histogram height preset threshold of the image is obtained based on the gray mean value, the gray variance, the distribution of the local histogram, and the size of the image. Further, under the limitation of the preset threshold, the histogram can be peak-cropped based on the preset threshold, the result is evenly distributed to all gray scales of the image, and then integral operation is performed to obtain the gray mapping curve, i.e. the CLAHE curve.

[0092] In the limited contrast adaptive histogram equalization algorithm, the histogram processing methods that can be implemented include but are not limited to: controlling the intensity of local tone mapping, limiting the height and maximum value of the histogram, determining the coefficient of the smoothing method, and adding a numerical value to each histogram. Based on this, the mapping parameters corresponding to the category of each pixel can be determined by combining the category of each pixel obtained by instance segmentation, and the tone mapping parameters of each image block can be determined by the mapping parameters corresponding to the category of each pixel. The CLAHE curve of each image block is generated according to the tone mapping parameters, thereby guiding the histogram processing of each image block.

[0093] The function of assisting local tone mapping by the mapping parameters of the category obtained by instance segmentation can better realize the mapping style and preference of different types of objects, and can better ensure the local effect.

[0094] In step S730, the mapping pixel values of the pixel point in the adjacent image blocks are interpolated to determine the target pixel value of the pixel point.

[0095] In this step, since each image block has a different tone mapping curve, it will cause the appearance of block effect. In order to improve the image quality, it is necessary to eliminate the block effect. When eliminating the block effect, interpolation processing can be used to execute, and the interpolation processing can be bilinear interpolation.

[0096] In some embodiments, for each pixel point, the mapping pixel values of the pixel point in the adjacent image blocks can be obtained. The adjacent image blocks refer to the image blocks adjacent to the image block to which the pixel point belongs. As shown in Figure 8 , the pixel point O is in the image block D, and the adjacent image blocks can be the image block A, the image block B and the image block C.

[0097] According to the manner in steps 710 and S720, the pixel point in each adjacent image block can be tone mapped by the tone mapping curve of each adjacent image block, that is, the pixel value of the pixel point is mapped by the tone mapping curve of each adjacent image block to obtain the corresponding mapping pixel value of the pixel point in each adjacent image block. Further, the mapping pixel value of the image block to which the pixel point belongs and the corresponding mapping pixel value of each adjacent image block are weighted and summed according to the weight parameter to realize interpolation processing, and the target pixel value of the pixel point is obtained according to the interpolation result.

[0098] The above steps are repeatedly executed to perform bilinear interpolation on the mapping pixel value of each pixel point in each image block to obtain the target pixel value of each pixel point to obtain the target image by tone mapping.

[0099] For example, as shown in Figure 8 , the gray value of the pixel point O in the image block D is , the gray value can also represent the pixel value. The neighboring image blocks of the image block D to which the pixel point O belongs can be the image block A, the image block B and the image block C. Assuming that the distance from the center point of the image block A to the pixel point O is x and y, the distance from the center point of the image block B to the pixel point O is 1-x and y, the distance from the center point of the image block C to the pixel point O is x and 1-y, and the distance from the center point of the image block D to the pixel point O is 1-x and 1-y. Assuming that the tone mapping curves of each image block, the neighboring image block and the image block to which the pixel point belongs are respectively . The pixel value of the pixel point O can be substituted into the tone mapping curves of the image block and the neighboring image block to which the pixel point belongs respectively to obtain the mapped pixel value of the pixel point, which can be represented as , , , . Next, the mapped pixel value of the pixel point in the image block and the mapped pixel value of the pixel point in the neighboring image block can be bilinearly interpolated to obtain the target pixel value of the pixel point. For example, the target pixel value of the pixel point O can be determined according to formula (5):

[0100] Formula (5)

[0101] wherein the weight parameter of the mapped pixel value of the image block is negatively related to the distance from the pixel point to the image block. The weight parameter of the mapped pixel value of the image block A is the product of the distance from the center point of the image block D to the pixel point O, the weight parameter of the mapped pixel value of the image block D is the product of the distance from the center point of the image block A to the pixel point O, the weight parameter of the mapped pixel value of the image block B is the product of the distance from the center point of the image block C to the pixel point O, and the weight parameter of the mapped pixel value of the image block C is the product of the distance from the center point of the image block B to the pixel point O.

[0102] In the embodiments of the present disclosure, the pixel value of the pixel point is mapped through the tone mapping curves of the neighboring image blocks of the image block to which the pixel point belongs, and the mapped pixel values of the multiple image blocks are bilinearly interpolated, so that the blocking effect caused by the different tone mapping curves between different image blocks can be eliminated, the transition between the image blocks is smooth and natural, there is no very conspicuous area on the whole image, and the smoothness and image quality are improved.

[0103] Figure 9 A flowchart for performing image processing is schematically shown in Figure 9 , which mainly includes the following steps:

[0104] In step S901, a to-be-processed image is acquired. The to-be-processed image is an RGB image.

[0105] In step S902, instance segmentation is performed on the image to be processed to obtain the category of each pixel.

[0106] In step S903, the image to be processed is divided into blocks to obtain a plurality of image blocks. Exemplarily, the number of image blocks can be pre-set to realize image blocking.

[0107] In step S904, the tone mapping parameter of each image block is determined according to the image block.

[0108] In step S905, the statistical information of the image block is obtained. The statistical information can be the mean value, maximum value, etc. of the image block.

[0109] In step S906, the tone mapping curve is generated in combination with the statistical information of each image block and the tone mapping parameter of each image block.

[0110] In step S907, each image block is tone mapped according to the tone mapping curve of each image block, and a target image is output.

[0111] Figure 10 The flowchart for obtaining a target image through tone mapping is shown in FIG. 10, and the process is described below with reference to FIG. 10. Figure 10 The process mainly includes the following steps:

[0112] In step S1001, a target image 1010 is obtained.

[0113] In step S1002, instance segmentation is performed on the target image to obtain the category of each pixel 1020.

[0114] In step S1003, the mapping parameter 1030 of each category 1020 is obtained.

[0115] In step S1004, the target image is divided into a plurality of image blocks 1040.

[0116] In step S1005, the tone mapping parameter 1050 of the image block is determined according to the mapping parameter of the category of all pixels contained in the image block 1040.

[0117] In step S1006, the tone mapping curve 1060 of each image block is determined in combination with the tone mapping parameter of the target image block.

[0118] In step S1007, the pixel value of each pixel in each image block is gamma mapped and expanded to a target value according to the tone mapping curve 1060 of each image block, and a mapped pixel value 1070 is obtained.

[0119] In step S1008, the pixel value of the pixel point is mapped according to the tone mapping curve of the adjacent image block to obtain a mapped pixel value 1080.

[0120] In step S1009, the mapped pixel value 1070 and the mapped pixel value 1080 are bilinearly interpolated to obtain a target pixel value 1090.

[0121] In step S1010, the target image 1000 is obtained according to the target pixel value of each pixel point.

[0122] The technical scheme in the embodiment of the present disclosure can determine the tone mapping curve of each image block by combining the mapping parameters of the categories obtained by instance segmentation, and can perform tone mapping on each image block by the tone mapping curve of each image block. Compared with the prior art, the technical scheme can determine the tone mapping curve of the image block by combining pixels of different categories, improve the accuracy and pertinence of the tone mapping curve, improve the accuracy of local tone mapping, and improve the application range and image quality. The results of semantic segmentation are used to guide the tone mapping curve, which better realizes local tone mapping. The mapping style and preference of different types of objects can be better realized, and the local effect can be better guaranteed. Two gamma correction coefficients are used to control the dark and bright areas of the image, which better realizes the regulation and control of the bright and dark areas. Different magnitudes of contrast enhancement processing can be accurately realized for different image blocks, and the effect and authenticity of local tone mapping can be improved.

[0123] The embodiment of the present disclosure provides an image tone mapping device. Figure 11 The image tone mapping device 1100 can include:

[0124] An image blocking module 1101 is configured to acquire a to-be-processed image and block the to-be-processed image to obtain a plurality of image blocks.

[0125] A mapping parameter acquisition module 1102 is configured to perform instance segmentation on the to-be-processed image to obtain the category of each pixel, and determine the mapping parameters corresponding to each category.

[0126] A mapping curve determination module 1103 is configured to combine the mapping parameters corresponding to the categories of all pixels in each image block to determine the tone mapping curve of each image block in the to-be-processed image.

[0127] A tone mapping module 1104 is configured to perform tone mapping on each image block based on the tone mapping curve of each image block to obtain a target image.

[0128] In an example embodiment of the present disclosure, the mapping curve determination module comprises: a mapping parameter determination module configured to determine a tone mapping parameter of each image block according to a mapping parameter corresponding to a category of all pixels contained in the image block in the image to be processed; and a curve generation module configured to generate a tone mapping curve of each image block according to the tone mapping parameter.

[0129] In an example embodiment of the present disclosure, the mapping parameter determination module comprises: a first determination module configured to determine the tone mapping parameter according to a mapping parameter of a category if all pixels in the image block comprise the category; and a second determination module configured to fuse mapping parameters of multiple categories to determine the tone mapping parameter of each image block if all pixels in the image block comprise the multiple categories.

[0130] In an example embodiment of the present disclosure, the second determination module comprises: a proportion determination module configured to determine a proportion of each category according to a number of pixels of each category contained in each image block, a mapping parameter of each category, and a total number of pixels corresponding to the image block; and a fusion module configured to fuse the proportion of each category to determine the tone mapping parameter of each image block.

[0131] In an example embodiment of the present disclosure, the curve generation module comprises: a generation control module configured to perform gamma mapping on a mean value of each image block and a maximum value in the image to be processed according to the tone mapping parameter of the target image block to generate the tone mapping curve of each image block.

[0132] In an example embodiment of the present disclosure, the tone mapping module comprises: a normalization mapping module configured to perform normalization mapping on a pixel value of a pixel point in each image block of the image to be processed to obtain a normalized pixel value of the pixel point; a gamma mapping module configured to perform gamma mapping on the normalized pixel value based on the tone mapping curve of the image block to which the pixel point belongs to obtain a mapping pixel value of the pixel point; and an interpolation module configured to perform interpolation processing on the mapping pixel values of the pixel point in adjacent image blocks to determine a target pixel value of the pixel point.

[0133] In an example embodiment of the present disclosure, the gamma mapping module comprises: a mapping expansion module configured to perform gamma mapping on the normalized pixel value based on the tone mapping curve of the image block to which the pixel point belongs, and expand the normalized pixel value after gamma mapping corresponding to the pixel point to a target value to determine the mapping pixel value.

[0134] In an example embodiment of the present disclosure, the interpolation module comprises: an adjacent image block mapping module, configured to map the pixel value of a pixel point based on the tone mapping curve of the adjacent image block of the pixel point, and obtain the mapping pixel value corresponding to each adjacent image block; and a bilinear interpolation module, configured to perform bilinear interpolation on the mapping pixel value of the image block and the mapping pixel value corresponding to each adjacent image block, and determine the target pixel value of the pixel point.

[0135] In an example embodiment of the present disclosure, the mapping parameter acquisition module comprises: a feature vector acquisition module, configured to perform feature extraction on the to-be-processed image multiple times through the multi-layer down-sampling network in the instance segmentation model, and obtain the feature vector of each layer of down-sampling network; and a feature fusion module, configured to obtain the corresponding reference feature vector according to the feature vector of each layer of network, and perform feature fusion on the reference feature vector and the feature vector through the multi-layer up-sampling network, so as to obtain the category of each pixel.

[0136] In an example embodiment of the present disclosure, the feature vector acquisition module comprises: a convolution module, configured to sequentially perform convolution operation on the to-be-processed image through each layer of down-sampling network in the instance segmentation model to obtain a convolution result; and a down-sampling module, configured to down-sample the convolution result to obtain a down-sampling result, and determine the down-sampling result as the feature vector of each layer of down-sampling network.

[0137] In an example embodiment of the present disclosure, the feature fusion module comprises: an up-sampling module, configured to up-sample the feature vector of each layer of down-sampling network through each layer of up-sampling network to obtain an up-sampling result; and a convolution module, configured to fuse the up-sampling result and the reference feature vector corresponding to each layer of down-sampling network to obtain fusion features, and perform convolution operation on the fusion features to obtain the category of each pixel.

[0138] It should be noted that the specific details of each part of the above image tone mapping device have been described in detail in the image tone mapping method part of the embodiment, and the details not disclosed can be referred to the embodiment content of the method part, and thus will not be described again.

[0139] The example embodiment of the present disclosure also provides an electronic device. The electronic device can be the terminal 101 described above. Generally, the electronic device can include a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above-mentioned image tone mapping method by executing the executable instructions.

[0140] In the following, the mobile terminal 1200 in Figure 12 will be taken as an example to illustrate the structure of the electronic device. Those skilled in the art should understand that, in addition to the components specially used for mobile purposes, Figure 12The configuration in the mobile terminal 1200 can also be applied to a stationary type device.

[0141] As shown in FIG. 12, the mobile terminal 1200 can include a processor 1201, a memory 1202, a bus 1203, a mobile communication module 1204, an antenna 1, a wireless communication module 1205, an antenna 2, a display screen 1206, a camera module 1207, an audio module 1208, a power module 1209, and a sensor module 1210. Figure 12 The processor 1201 can include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit), etc. The image denoising processing method in the present exemplary embodiment can be executed by the AP, the GPU, or the DSP, and when the method involves neural network related processing, the NPU can be used to execute the method, for example, the NPU can load neural network parameters and execute neural network related algorithm instructions. Exemplarily, the ISP in the processor can divide a to-be-processed image into a plurality of image blocks; determine a global tone mapping coefficient of the to-be-processed image, and determine local tone mapping coefficients of the image blocks; according to the global tone mapping coefficient and the local tone mapping coefficients, interpolate to obtain a target coefficient matrix corresponding to each pixel point of the to-be-processed image; and according to the target coefficient matrix, correct parameters of each pixel point in the to-be-processed image to generate a target image corresponding to the to-be-processed image.

[0142]

[0143] ​An encoder can encode (i.e., compress) an image or video to reduce data size for storage or transmission. A decoder can decode (i.e., decompress) encoded data of an image or video to restore the image or video data. The mobile terminal 1200 can support one or more encoders and decoders, such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap), and the like for image formats, and MPEG (Moving Picture Experts Group) 1, MPEG 10, H.1063, H.1064, HEVC (High Efficiency Video Coding), and the like for video formats.

[0144] The processor 1201 can form a connection with the memory 1202 or other components through the bus 1203.

[0145] The memory 1202 can be used to store computer executable program codes including instructions. The processor 1201 executes various functional applications and data processing of the mobile terminal 1200 by running the instructions stored in the memory 1202. The memory 1202 can also store application data, such as storing image, video, and the like.

[0146] The communication function of the mobile terminal 1200 can be implemented through the mobile communication module 1204, an antenna 1, a wireless communication module 1205, an antenna 2, a modem processor, and a baseband processor, and the like. The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1204 can provide a mobile communication solution applied to the mobile terminal 1200, such as 3G, 4G, 5G, and the like. The wireless communication module 1205 can provide a wireless communication solution applied to the mobile terminal 1200, such as wireless local area network, Bluetooth, near field communication, and the like.

[0147] The display screen 1206 is used to implement a display function, such as displaying a user interface, an image, a video, and the like. The camera module 1207 is used to implement a photographing function, such as photographing an image, a video, and the like, and the camera module can include a color temperature sensor array. The audio module 1208 is used to implement an audio function, such as playing audio, collecting voice, and the like. The power module 1209 is used to implement a power management function, such as charging a battery, supplying power to the device, monitoring a battery state, and the like. The sensor module 1210 can include one or more sensors to implement a corresponding sensing and detecting function. For example, the sensor module 1210 can include an inertial sensor for detecting a motion pose of the mobile terminal 1200 and outputting inertial sensing data.

[0148] It should be noted that the embodiments of the present disclosure also provide a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device.

[0149] The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device.

[0150] The computer readable storage medium can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0151] The computer readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.

[0152] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0153] In addition, the above figures are only schematic illustrations of the processes included in the methods according to the example embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of the processes. In addition, it is also easy to understand that the processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0154] It should be noted that, although several modules or units for devices for action execution are mentioned in the above detailed description, such division is not mandatory. Indeed, according to an embodiment of the disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into embodied by multiple modules or units.

[0155] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the concepts disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It is to be understood that the disclosure is not to be limited to the precise construction herein disclosed and that various modifications and changes can be effected therein by those skilled in the art without departing from the scope of the disclosure. The scope of the disclosure is to be afforded the broadest interpretation so as to encompass all compatible modifications and enhancements.

Claims

1. An image tone mapping method, characterized by, The method comprises: acquiring a to-be-processed image, and dividing the to-be-processed image into a plurality of image blocks; performing instance segmentation on the to-be-processed image to obtain the category of each pixel, and determining the mapping parameters corresponding to each category; combining the mapping parameters corresponding to the category of all pixels in each image block to determine the tone mapping curve of each image block in the to-be-processed image; performing tone mapping on each image block based on the tone mapping curve of each image block to obtain a target image; wherein the combination of the mapping parameters corresponding to the category of all pixels in each image block to determine the tone mapping curve of each image block in the to-be-processed image comprises: determining the tone mapping parameters of each image block according to the mapping parameters corresponding to the category of all pixels contained in each image block in the to-be-processed image; calculating the ratio of the mean value of each image block to the maximum value of the gray value of the to-be-processed image before mapping to obtain the ratio of each image block; and performing fusion based on the ratio of each image block and the tone mapping parameters of the target image block to generate the tone mapping curve of each image block.

2. The image tone mapping method of claim 1, wherein, The determination of the tone mapping parameters of each image block according to the mapping parameters corresponding to the category of all pixels contained in each image block in the to-be-processed image comprises: if all the pixels in the image block include one category, determining the tone mapping parameters according to the mapping parameters of the category; if all the pixels in the image block include multiple categories, fusing the mapping parameters of the multiple categories to determine the tone mapping parameters of each image block.

3. The image tone mapping method of claim 2, wherein, The fusion of the mapping parameters of the multiple categories to determine the tone mapping parameters of each image block comprises: determining the proportion of each category according to the number of pixels of each category contained in each image block, the mapping parameters of each category, and the total number of pixels corresponding to the image block; fusing the proportion of each category to determine the tone mapping parameters of each image block.

4. The image tone mapping method of claim 1, wherein, The tone mapping of each image block based on the tone mapping curve of each image block to obtain a target image comprises: performing normalization mapping on the pixel value of a pixel point in each image block of the to-be-processed image to obtain the normalized pixel value of the pixel point; performing gamma mapping on the normalized pixel value based on the tone mapping curve of the image block to which the pixel point belongs to obtain the mapping pixel value of the pixel point; performing interpolation processing on the mapping pixel values of the pixel point in adjacent image blocks to determine the target pixel value of the pixel point.

5. The image tone mapping method of claim 4, wherein, The gamma mapping of the normalized pixel value based on the tone mapping curve of each image block to obtain the mapping pixel value of the pixel point comprises: performing gamma mapping on the normalized pixel value based on the tone mapping curve of the image block to which the pixel point belongs, and expanding and mapping the normalized pixel value after gamma mapping corresponding to the pixel point to a target value to determine the mapping pixel value.

6. The image tone mapping method of claim 4, wherein, The interpolation processing of the mapping pixel values of the pixel point in adjacent image blocks to determine the target pixel value of the pixel point comprises: performing mapping on the pixel value of the pixel point based on the tone mapping curve of the adjacent image blocks of the pixel point to obtain the mapping pixel value corresponding to each adjacent image block; The target pixel value of the pixel point is determined by bilinear interpolation of the mapped pixel value of the image block and the mapped pixel value corresponding to each adjacent image block.

7. The image tone mapping method of claim 1, wherein, The method comprises: The image is segmented by the instance segmentation model to obtain the category of each pixel. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained.

8. The image tone mapping method of claim 7, wherein, The feature vector is fused with the reference feature vector through the multi-layer upsampling network to obtain the category of each pixel. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained.

9. The image tone mapping method of claim 7, wherein, The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained.

10. An image tone mapping apparatus, characterized by, The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained.

11. An electronic device, comprising: The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of the network is obtained, and the reference feature vector is obtained. The feature vector of each layer of

Images