Image processing method and device, equipment and storage medium

Abstract

The invention provides an image processing method and device, equipment and a storage medium. The method comprises the steps that a first segmentation result of a to-be-processed image is acquired, the first segmentation result comprises one or more first segmentation areas and first category labels of the first segmentation areas, and the first category labels are used for identifying scene categories of the corresponding segmentation areas; selecting one or more target segmentation areas from the one or more first segmentation areas according to the first category labels of the first segmentation areas; determining a first tone mapping curve according to the pixel value distribution condition of the one or more target segmentation areas; and performing tone mapping processing on the to-be-processed image according to the first tone mapping curve, and determining a target image of the to-be-processed image.

Description

Technical Field

[0001] The present application relates to electronic technology, and is related to but not limited to image processing methods and devices, equipment, and storage media. Background Art

[0002] Technological advances are driving changes in lifestyles. With the rise of short video platforms and the upgrade of mobile phone camera modules, people are no longer limited to sharing their daily lives in the form of text and pictures on social software. Sharing personal lives in the form of videos has become a new way for the younger generation to create and share.

[0003] Users' attention to video effects has also gradually shifted from basic aspects such as high definition and image stability to stylized characteristics such as capturing a more natural and realistic world. The most important aspect is the tone effect of the video. Summary of the invention

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: obtaining a first segmentation result of an image to be processed, the first segmentation result comprising one or more first segmentation regions and a first category label of the first segmentation region, the first category label being used to identify a scene category of the corresponding segmentation region; selecting one or more target segmentation regions from the one or more first segmentation regions according to the first category label of the first segmentation region; determining a first tone mapping curve according to a pixel value distribution of the one or more target segmentation regions; and performing tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

[0005] In a second aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition module, configured to acquire a first segmentation result of an image to be processed, the first segmentation result comprising one or more first segmentation regions and a first category label of the first segmentation region, the first category label being used to identify a scene category of the corresponding segmentation region; a first determination module, configured to select one or more target segmentation regions from the one or more first segmentation regions according to the first category label of the first segmentation region; a second determination module, configured to determine a first tone mapping curve according to a pixel value distribution of the one or more target segmentation regions; and a third determination module, configured to perform tone mapping processing on the image to be processed according to the first tone mapping curve, and determine a target image of the image to be processed.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, the method described in the first aspect is implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when executed by a processor or an electronic device.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor or an electronic device, implements the method described in the first aspect of the present application.

[0009] In a sixth aspect, an embodiment of the present application provides a computer program, which enables a processor or an electronic device to execute the method described in the first aspect.

[0010] It can be understood that in the embodiment of the present application, the first segmentation result of the image to be processed is obtained, and the first segmentation result includes not only the first segmentation area obtained by segmenting the image to be processed, but also the first category label of the first segmentation area. According to the scene category (that is, the first category label) of each first segmentation area, one or more target segmentation areas are selected therefrom, and the first tone mapping curve is determined according to the pixel value distribution of the one or more target segmentation areas. It can be seen that the first tone mapping curve is closely related to the actual image content and scene category of the image to be processed. Therefore, tone mapping processing is performed on the image to be processed based on the tone mapping curve to determine the target image of the image to be processed, which can reduce the probability of tone flipping in the target image, so that the obtained target image is more realistic and natural, and has a tone level that is comfortable to look at.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0014] Figure 1 A schematic diagram of the implementation flow of the image processing method provided in the embodiment of the present application;

[0015] Figure 2 An example diagram of a segmentation diagram of an image to be processed provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of a further implementation flow of step 102 provided in an embodiment of the present application;

[0017] Figure 4 A schematic diagram of a further implementation process of step 104 provided in the embodiment of the present application Figure 1 ;

[0018] Figure 5 An example diagram of the impact of the segmentation result provided in the embodiment of the present application on the target image;

[0019] Figure 6 A schematic diagram of a further implementation flow of step 402 provided in an embodiment of the present application;

[0020] Figure 7 A schematic diagram of a further implementation flow of step 601 provided in an embodiment of the present application;

[0021] Figure 8 A schematic diagram of a further implementation flow of step 705 provided in an embodiment of the present application;

[0022] Fig. 9 A schematic diagram of a further implementation flow of step 602 provided in an embodiment of the present application;

[0023] Fig.10 A schematic diagram of a further implementation flow of step 603 provided in an embodiment of the present application;

[0024] Fig.11 A schematic diagram of a further implementation process of step 104 provided in the embodiment of the present application Figure 2 ;

[0025] Fig.12 An example diagram of an image to be processed and a corresponding grayscale histogram provided in an embodiment of the present application;

[0026] Fig.13 A schematic diagram of a further implementation flow of step 1102 provided in an embodiment of the present application;

[0027] Fig.14 A schematic diagram of a further implementation process of step 104 provided in the embodiment of the present application Figure 3 ;

[0028] Fig.15 A schematic diagram of a further implementation flow of step 1403 provided in an embodiment of the present application;

[0029] Fig.16A schematic diagram of a further implementation process of step 104 provided in the embodiment of the present application Figure 4 ;

[0030] Fig.17 An example diagram of the weights of the segmentation map and the corresponding segmentation boundary area provided in the embodiment of the present application;

[0031] Fig.18 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0032] Fig.19 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0035] In the following description, reference is made to “some embodiments”, “this embodiment”, “embodiments of the present application” and examples, etc., which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] The descriptions such as “first, second, third” that appear in the embodiments of the present application do not have specific meanings and are merely for the convenience of clearly describing the embodiments of the present application. The descriptions such as “first, second, third” cannot constitute any limitation on the embodiments of the present application.

[0037] An embodiment of the present application provides an image processing method.

[0038] Figure 1 The following is a schematic diagram of the implementation process of the image processing method provided in the embodiment of the present application. Figure 1 As shown, the method includes the following steps 101 to 104:

[0039] Step 101, obtaining a first segmentation result of an image to be processed, wherein the first segmentation result includes one or more first segmented regions and first category labels of the first segmented regions, wherein the first category labels are used to identify a scene category of the corresponding segmented regions;

[0040] Step 102, selecting one or more target segmented regions from the one or more first segmented regions according to the first category labels of the first segmented regions;

[0041] Step 103, determining a first tone mapping curve according to the pixel value distribution of the one or more target segmented areas;

[0042] Step 104: Perform tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

[0043] It can be understood that in the embodiment of the present application, the first segmentation result of the image to be processed is obtained, and the first segmentation result includes not only the first segmentation area obtained by segmenting the image to be processed, but also the first category label of the first segmentation area. According to the scene category (that is, the first category label) of each first segmentation area, one or more target segmentation areas are selected therefrom, and the first tone mapping curve is determined according to the pixel value distribution of the one or more target segmentation areas. It can be seen that the first tone mapping curve is closely related to the actual image content and scene category of the image to be processed. Therefore, tone mapping processing is performed on the image to be processed based on the tone mapping curve to determine the target image of the image to be processed, which can reduce the probability of tone flipping in the target image, so that the obtained target image is more realistic and natural, and has a tone level that is comfortable to look at.

[0044] The following describes further optional implementations and related terms of each of the above steps.

[0045] Step 101: Obtain a first segmentation result of the image to be processed, where the first segmentation result includes one or more first segmented regions and first category labels of the first segmented regions, where the first category labels are used to identify a scene category of the corresponding segmented region.

[0046] In the embodiment of the present application, there is no limitation on the method for determining the first segmentation result. In short, segmenting the image to be processed can obtain one or more first segmented regions of the image to be processed. The image to be processed can be segmented by an image segmentation algorithm or a pre-trained image segmentation network (i.e., a pre-trained AI model).

[0047] It can be understood that the first category label is used to identify the scene category of the corresponding first segmented area, and the first category labels corresponding to different first segmented areas may be different. Figure 2 This is an example diagram of a segmentation diagram of an image to be processed provided in an embodiment of the present application; Figure 2As shown, a certain image to be processed is segmented into five first segmented areas, wherein the first category label of the first segmented area 1 identifies the scene category of the area as sky, the first category label of the first segmented area 2 identifies the scene category of the area as green plants, the first category label of the first segmented area 3 identifies the scene category of the area as portrait, the first category label of the first segmented area 4 identifies the scene category of the area as street, and the first category label of the first segmented area 5 identifies the scene category of the area as building. Of course, Figure 2 The first segmentation result is merely exemplified and does not limit the scene category and the number of first segmented regions described in the present application. In addition, in the embodiment of the present application, an image composed of one or more first segmented regions obtained by segmenting the image to be processed can also be referred to as a segmentation map of the image to be processed.

[0048] Step 102: Select one or more target segmented regions from the one or more first segmented regions according to the first category labels of the first segmented regions.

[0049] In the embodiment of the present application, there is no limitation on the further implementation of step 102. The first segmented region with a specific category label in the first category label can be used as the target segmented region, or the first segmented region with a specific category label in the first category label and a size-related parameter greater than or equal to the first threshold can be used as the target segmented region. In short, there is no limitation on the screening condition for screening out the target segmented region based on the first category label.

[0050] In some embodiments, the target segmentation region is a non-portrait scene, and the size-related parameter of the target segmentation region is greater than or equal to a first threshold. In other embodiments, the target segmentation region is a non-portrait scene, and the target segmentation region is the first K first segmentation regions with the largest size-related parameters among the one or more first segmentation regions, where K is a preset value, for example, K is 1 or greater than 1.

[0051] In the embodiment of the present application, there is no limitation on the size-related parameter of the first segmented area, and the size-related parameter may be the number of pixels contained in the first segmented area, or the size proportion of the first segmented area in the image to be processed, etc.

[0052] For step 102, illustratively, in some embodiments, Figure 3 As shown, step 102 may include the following steps 301 to 303:

[0053] Step 301, determining the main scene category of the image to be processed according to the first category label of the first segmented area;

[0054] Step 302, determining a first segmented area corresponding to a non-main scene category from the one or more first segmented areas;

[0055] Step 303: Select one or more target segmentation regions from the first segmentation regions corresponding to the non-main scene categories.

[0056] It is understandable that Figure 3 In the method for determining the target segmentation area shown, one or more target segmentation areas are selected from the first segmentation areas corresponding to the non-main scene categories, and the first tone mapping curve is further determined based on this; in this way, since the first segmentation area corresponding to the main scene category does not participate in the determination of the first tone mapping curve, the influence of the first tone mapping curve on the mapping of the first segmentation area belonging to the main scene category in the processed image is weakened, thereby ensuring that the first segmentation area corresponding to the main scene category still has good authenticity after being tone mapped, thereby improving the overall visual effect of the target image.

[0057] For example, assuming that the main scene category is a portrait scene, the first segmented area with the portrait does not participate in the determination of the first tone mapping curve. The main considerations for this design are: first, the inventors of the present application found in research and analysis that if the first segmented area with the portrait participates in the first tone mapping curve, the target image finally obtained may have an unrealistic tone at the junction of the portrait and the non-portrait. Therefore, the first tone mapping area (that is, the basic tone of the entire image) is determined based on the non-portrait area, and there will be no obvious tone splitting at the junction of the portrait area and the non-portrait area in the target image determined based on this; second, considering that portraits may have different scales, and different portraits have their own characteristics (such as different light and shadow and skin color, etc.), and users are also sensitive to the beauty of portraits, therefore, the portrait area does not participate in the determination of the first tone mapping curve, so as not to perform particularly prominent mapping processing on the portrait area, thereby ensuring the beauty and authenticity of the portrait area.

[0058] The following describes further optional implementations and related terms of each of the above steps.

[0059] Step 301: Determine the main scene category of the image to be processed according to the first category label of the first segmented area.

[0060] Step 302: Determine a first segmented area corresponding to a non-main scene category from the one or more first segmented areas.

[0061] In some embodiments, step 302 may further include: when the main scene category is a portrait scene, determining a first segmented area corresponding to a non-portrait scene from the one or more first segmented areas.

[0062] It can be understood that there may be one or more first segmented regions corresponding to the non-main scene category selected in step 302, and the number of first segmented regions corresponding to the non-main scene category is related to the actual segmentation result of the image to be processed.

[0063] Step 303: Select one or more target segmentation regions from the first segmentation regions corresponding to the non-main scene categories.

[0064] Further, in some embodiments, step 303 includes: determining a size-related parameter of a first segmented area corresponding to the non-main scene category; and selecting one or more target segmented areas whose size-related parameters satisfy a first condition from the first segmented areas corresponding to the non-main scene category; wherein the first condition includes that the size-related parameter is greater than or equal to a first threshold, or the first condition includes the first K maximum values, K being greater than or equal to 1.

[0065] That is, the one or more target segmented regions are segmented regions whose size-related parameters are greater than or equal to the first threshold value among the first segmented regions corresponding to the non-main scene category.

[0066] Alternatively, the one or more target segmented regions are the first K segmented regions with the largest size-related parameters among the first segmented regions corresponding to the non-main scene category.

[0067] It can be understood that determining the first tone mapping curve based on the target segmentation area whose size-related parameters meet the first condition is beneficial to improving the globality of the first tone-related curve, and then based on this, tone mapping processing is performed on the image to be processed to determine the target image of the image to be processed, which is beneficial for the target image to contain better tonal levels.

[0068] Step 103: determining a first tone mapping curve according to the pixel value distribution of the one or more target segmented regions.

[0069] In the embodiment of the present application, there is no limitation on the further implementation of step 103. In short, the first tone mapping curve is determined based on the distribution of pixel values ​​of the one or more target segmented regions.

[0070] Exemplarily, for step 103, in some embodiments, step 103 may include: performing statistics on the pixel value distribution of the one or more target segmentation areas to determine a first cumulative histogram of the one or more target segmentation areas; and determining the first tone mapping curve based on the first cumulative histogram.

[0071] Further, in some embodiments, the counting of the pixel value distribution of the one or more target segmentation areas to determine the first cumulative histogram of the one or more target segmentation areas includes: counting the pixel value distribution of the one or more target segmentation areas to obtain a grayscale histogram; accumulating the grayscale histogram to obtain a second cumulative histogram; normalizing the second cumulative histogram to obtain the first cumulative histogram; wherein each element of the second cumulative histogram is the cumulative sum of the grayscale histogram from the lowest grayscale level to the current grayscale level. This means that for each grayscale level i, the value of the second cumulative histogram is the sum of the number of pixels with grayscale levels less than or equal to i.

[0072] For the step of "determining the first tone mapping curve according to the first cumulative histogram", in a possible implementation, the first cumulative histogram may be converted into the first tone mapping curve according to a predefined mapping rule between the cumulative probability of gray levels and the mapped pixel values. That is, the cumulative probability corresponding to the gray levels of the first cumulative histogram and the pixel values ​​corresponding to the gray levels of the first tone mapping curve have a predefined mapping relationship.

[0073] Step 104: Perform tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

[0074] In the embodiment of the present application, there is no limitation on the further implementation of step 104, which may be solution 1, solution 2, solution 3, solution 4 or solution 5 described below.

[0075] Among them, in scheme 1, for step 104, further, in some embodiments, as Figure 4 As shown, step 104 may include the following steps 401 and 402:

[0076] Step 401, determining a first mapping strength according to the first category labels of the one or more first segmented regions;

[0077] Step 402: Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the first mapping intensity to determine a target image of the image to be processed.

[0078] Understandably, in Figure 4In the method for determining the target image shown, the tone mapping processing of the image to be processed is not only based on the first tone mapping curve, but also based on the first mapping intensity. The first mapping intensity is determined according to the actual scene category (i.e., the first category label) of the one or more first segmented regions; this is beneficial for obtaining a more realistic and natural target image with better tone levels.

[0079] The following describes further optional implementations and related terms of each of the above steps.

[0080] Step 401: determine a first mapping strength according to first category labels of the one or more first segmented regions.

[0081] In the embodiment of the present application, there is no limitation on the further implementation of step 401. In one possible implementation, the first mapping strength may be an average or weighted average of the mapping strengths corresponding to the first category labels of the one or more first segmented regions.

[0082] In another possible implementation, step 401 may include: determining a main scene category of the image to be processed according to the first category labels of the one or more first segmented regions; and determining a corresponding first mapping intensity according to the main scene category.

[0083] It should be understood that different main scene categories correspond to different first mapping intensities. The first mapping intensity corresponding to the main scene category of the image to be processed may be determined according to a predefined mapping relationship between the main scene category and the first mapping intensity.

[0084] In one possible implementation, the first mapping intensity of the main scene category of portrait is less than the first mapping intensity of the main scene category of non-portrait. The main consideration for this design is that users are more sensitive to the appearance of portraits. Therefore, if a strong tonal enhancement is performed on the portrait, the portrait area may appear too bright or too dark when mapped, or the color may be abnormal (such as oversaturation), which are all undesirable.

[0085] Step 402: Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the first mapping intensity to determine a target image of the image to be processed.

[0086] It should be noted that, in a further implementation of step 402, that is, in the process of obtaining the target image, the accuracy of the first segmented area may be checked first, or may not be checked, and this application does not impose any limitation on this.

[0087] However, the inventors of the present application have found in their research and analysis that a certain first segmented area may have a local misdetection problem, and the local misdetection problem may further lead to the problem of local brightness blocks in the final target image. Figure 5 As shown, the segmentation result pointed by the arrow in the right figure should not be segmented as belonging to the building, which is a local misdetection problem. The left figure is the target image obtained based on the segmentation result of the right figure. It can be seen that the circled area in the target image is too bright, that is, there is a local brightness color block problem.

[0088] In view of this, in some embodiments, Figure 6 As shown, step 402 includes the following steps 601 to 603:

[0089] Step 601, correcting the pixel values ​​of the first segmented area in the first segmentation result to obtain a second segmented area;

[0090] Step 602, determining a local mask corresponding to the segmented region according to the second segmented region and the first category label;

[0091] Step 603: Perform tone mapping processing on the image to be processed according to the first tone mapping curve, the first mapping intensity, and the local mask to determine a target image of the image to be processed.

[0092] In step 601, the pixel values ​​of the first segmented area in the first segmentation result are corrected to obtain the second segmented area.

[0093] Furthermore, in some embodiments, Figure 7 As shown, step 601 includes the following steps 701 to 705:

[0094] Step 701, determining a first grayscale image of the image to be processed;

[0095] Step 702: scaling the first grayscale image to a segmentation image size corresponding to the first segmentation result to obtain a second grayscale image.

[0096] It should be understood that the segmentation map size refers to the size of the entire segmentation map composed of the first segmentation area in the first segmentation result. The segmentation map size can be the size of the entire segmentation map obtained when the processed image is segmented by an image segmentation algorithm or a pre-trained image segmentation network (that is, a pre-trained AI model). The segmentation map size can be a pre-defined size.

[0097] Step 703: Perform edge detection on the second grayscale image to determine edge pixels of the second grayscale image.

[0098] In the embodiment of the present application, there is no limitation on the edge detection algorithm used to implement step 703. For example, the Canny algorithm, Sobel algorithm, Laplacian algorithm, Roberts algorithm, Prewitt algorithm or Scharr algorithm may be used.

[0099] Step 704, determining the pixel to be corrected in the first segmented area according to the pixel value of the edge pixel in the second grayscale image; the difference between the pixel value of the pixel to be corrected and the pixel value of the co-located edge pixel is greater than or equal to a second threshold;

[0100] Step 705: Correct the pixels to be corrected in the first segmented area to obtain a second segmented area.

[0101] Furthermore, in some embodiments, Figure 8 As shown, step 705 includes the following steps 801 and 802:

[0102] Step 801: Search the second grayscale image for a reference pixel having a pixel value similar to that of the pixel to be corrected.

[0103] Exemplarily, in some embodiments, step 801 may include: determining a similarity parameter between the pixel value of the pixel to be corrected and the pixel value of the pixel in the second grayscale image; and selecting a reference pixel whose similarity parameter satisfies a similarity condition from the second grayscale image. For example, the similarity parameter is a difference in pixel values, and the absolute value of the difference between the pixel value of the reference pixel and the pixel value of the pixel to be corrected is less than or equal to a third threshold.

[0104] Step 802: Correct the pixel value of the corresponding pixel to be corrected according to the pixel value of the reference pixel to obtain a second segmented area.

[0105] Further, in some embodiments, step 802 includes: determining an average value or a weighted average value of the pixel values ​​of the reference pixels; and correcting the pixel values ​​of the corresponding pixels to be corrected according to the average value or the weighted average value to obtain a second segmented area. For example, the pixel values ​​of the corresponding pixels to be corrected are replaced / corrected to the average value or the weighted average value of the pixel values ​​of the reference pixels.

[0106] Further, in some other embodiments, step 802 includes: determining an average value or a weighted average value of the pixel value of the reference pixel and the pixel value of the pixel to be corrected; and correcting the pixel value of the corresponding pixel to be corrected according to the average value or the weighted average value to obtain a second segmented area. For example, the pixel value of the corresponding pixel to be corrected is replaced / corrected to the average value or the weighted average value of the pixel value of the reference pixel and the pixel value of the pixel to be corrected (before correction).

[0107] In step 602, a local mask corresponding to the segmented region is determined according to the second segmented region and the first category label.

[0108] It can be understood that the local mask corresponding to the segmented area is determined based on the actual scene category and the actual image content of the segmented area. In this way, the value of the local mask is more consistent with the actual image content of the corresponding segmented area, which is beneficial to obtaining a better tone mapping processing result of the image to be processed, so that the final target image is more realistic and natural, and has a better tone level.

[0109] In some embodiments, Fig. 9 As shown, step 602 includes the following steps 901 to 903:

[0110] Step 901: Determine an adjustment coefficient of a pixel value corresponding to a segmented area according to the first category label.

[0111] In the embodiment of the present application, different first category labels correspond to respective adjustment coefficients. A further implementation of step 901 may be: searching for the adjustment coefficient corresponding to the first category label from a predefined mapping relationship between the category label and the adjustment coefficient.

[0112] Step 902: adjusting the pixel value of the corresponding second segmented area according to the adjustment coefficient to obtain a third segmented area.

[0113] In a possible implementation, the adjustment coefficient may be multiplied by the pixel value of the corresponding second segmented area to obtain the third segmented area. Of course, the present application does not limit how to calculate the relationship between the adjustment coefficient and the pixel value of the second segmented area. In short, the pixel value of the second segmented area needs to be adjusted according to the adjustment coefficient to obtain the third segmented area.

[0114] Step 903: determine a corresponding local mask according to the third segmented area.

[0115] Further, in some embodiments, step 903 includes: scaling the first segmentation map composed of the third segmentation areas corresponding to the one or more first segmentation areas to the size of the image to be processed to obtain a second segmentation map; and normalizing the pixel values ​​of the second segmentation map to obtain a local mask corresponding to the third segmentation area.

[0116] Wherein, for step 603, tone mapping processing is performed on the image to be processed according to the first tone mapping curve, according to the first mapping intensity and according to the local mask to determine a target image of the image to be processed.

[0117] In the embodiment of the present application, there is no limitation on the further implementation method of step 603, that is, there is no limitation on how to use the first tone mapping curve, the first mapping intensity and the local mask to realize the tone mapping processing of the image to be processed, and there is no limitation on the order of operations based on the first tone mapping curve, the first mapping intensity and the local mask. In short, this information is required as input information for the tone mapping processing of the image to be processed.

[0118] Exemplarily, in some embodiments, step 603 may further include: first mapping the pixel values ​​of the image to be processed using a first tone mapping curve to obtain a first mapping image, and then adjusting the pixel values ​​of the first mapping image using a first mapping intensity to obtain a second mapping image (for example, multiplying the first mapping intensity by the pixel value of the first mapping image to obtain the second mapping image); finally, determining the target image of the image to be processed based on the second mapping image and the local mask (multiplying the pixel value of each pixel position of the second mapping image by the value of the same pixel position in the local mask, and determining the target image based on this).

[0119] For example, in some other embodiments, Fig.10 As shown, step 603 may further include the following steps 1001 and 1002:

[0120] Step 1001: determine a second tone mapping curve according to the first tone mapping curve and the first mapping intensity.

[0121] It can be understood that the first tone mapping curve includes a grayscale or pixel value and a new pixel value corresponding to the grayscale or pixel value (i.e., the pixel value after tone mapping). For step 1001, in a possible implementation, the first mapping intensity can be applied to the pixel value in the first tone mapping curve, so as to adjust the pixel value in the first tone mapping curve, and the second tone mapping curve is determined based on this, for example, the second tone mapping curve is the adjusted first tone mapping curve, and of course, the second tone mapping curve can also be a curve obtained by further processing the adjusted first tone mapping curve, which is not limited in this application.

[0122] Step 1002: Perform tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine a target image of the image to be processed.

[0123] In a possible implementation, the result obtained by the tone mapping process can be directly used as the target image. For example, the target image of the image to be processed can be determined by the following formula:

[0124] Target image = Global[input]*local mask+(1-local mask)*input;

[0125] Among them, Global[] refers to the second tone mapping curve, input refers to the image to be processed, and local mask refers to a local mask image composed of all local masks corresponding to the image to be processed.

[0126] In another possible implementation, the result obtained by the tone mapping (which we call an intermediate image) can be further processed (such as boundary fusion processing) to obtain the target image; that is, step 1002 further includes: according to the second tone mapping curve and the local mask, tone mapping processing is performed on the image to be processed to obtain an intermediate image; and the boundary area of ​​the second segmented area in the intermediate image is fused to determine the target image.

[0127] For example, the intermediate image can be determined by the following formula:

[0128] Intermediate image = Global[input]*local mask+(1-local mask)*input;

[0129] Among them, Global[] refers to the second tone mapping curve, input refers to the image to be processed, and local mask refers to a local mask image composed of all local masks corresponding to the image to be processed.

[0130] For step 104, tone mapping processing is performed on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

[0131] As mentioned above, in the embodiments of the present application, there is no limitation on the further implementation of step 104, which may be the scheme 1 described above and the scheme 2, scheme 3, scheme 4 or scheme 5 described below.

[0132] Among them, in scheme 2, for step 104, further, in other embodiments, as Fig.11 As shown, step 104 may include the following steps 1101 and 1102:

[0133] Step 1101, determining a second mapping intensity according to a brightness change of the image to be processed;

[0134] Step 1102: Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the second mapping intensity to determine a target image of the image to be processed.

[0135] Understandably, in Fig.11 The method for determining the target image shown in the figure is not only based on the first tone mapping curve, but also based on the second mapping intensity for the tone mapping processing of the image to be processed. The second mapping intensity is determined according to the actual brightness change of the image to be processed; thus, it is beneficial to obtain a more realistic and natural target image with better tone levels.

[0136] The following describes further optional implementations and related terms of each of the above steps.

[0137] Step 1101: determine a second mapping intensity according to a brightness change of the image to be processed.

[0138] In a possible implementation, different degrees of brightness change can be divided, and the second mapping intensities corresponding to the different degrees of brightness change can be predefined. In the embodiment of the present application, there is no limitation on the specific division of brightness change into several levels / scenes. Exemplarily, in some embodiments, the brightness change of the image to be processed includes a high dynamic range scene, a low dynamic range scene, or a mixed scene of the high dynamic range scene and the low dynamic range scene; wherein the second mapping intensity corresponding to the high dynamic range scene is greater than the second mapping intensity corresponding to the mixed scene, and the second mapping intensity corresponding to the mixed scene is greater than the second mapping intensity corresponding to the low dynamic range scene.

[0139] It can be understood that different brightness changes correspond to different second mapping intensities. For example, the second mapping intensity corresponding to the high dynamic range scene is greater than the second mapping intensity corresponding to the mixed scene, and the second mapping intensity corresponding to the mixed scene is greater than the second mapping intensity corresponding to the low dynamic range scene. The main consideration for this design is: the reason why the mapping intensity corresponding to high dynamic is large is that for high dynamic range scenes, the target image to be processed should have a rich human eye sensory range as much as possible, so as to ensure better contrast of the picture; and for low dynamic range scenes, the mapping intensity is reduced as much as possible, in order to reduce the probability of brightness flipping in the target image; in short, the ultimate goal is to enhance the color perception of the target image while making the final target image conform to the real scene and have a natural color tone.

[0140] Regarding how to determine the brightness change of the image to be processed, in a possible implementation, the proportion of different brightness ranges in the image to be processed to the entire image can be counted, and based on this, the brightness change level / scene of the image to be processed can be determined. Fig.12As shown, the grayscale histogram 1201 is a statistical result of the brightness change of the image 1211 to be processed, the grayscale histogram 1202 is a statistical result of the brightness change of the image 1212 to be processed, and the grayscale histogram 1203 is a statistical result of the brightness change of the image 1213 to be processed. Fig.12 As shown, if a specific brightness value (such as 128) is used as a boundary, if the proportion of the number of pixels with brightness values ​​less than the specific brightness value is greater than a fourth threshold value (such as 40%), and the proportion of the number of pixels with brightness values ​​greater than the specific brightness value is greater than the fourth threshold value, it is determined that the brightness change of the image to be processed belongs to a high dynamic range scene (such as the image to be processed 1211 corresponding to the grayscale histogram 1201); if the proportion of the number of pixels with brightness values ​​in the preset brightness value range is greater than the fifth threshold value (such as 50%), it is determined that the brightness change of the image to be processed belongs to a low dynamic range scene (such as the image to be processed 1212 corresponding to the grayscale histogram 1202); wherein, for example, the specific brightness value belongs to the preset brightness value range. Otherwise, if the brightness change of the image to be processed does not belong to a high dynamic range scene or a low dynamic range scene, it is determined that the brightness change of the image to be processed belongs to a mixed scene of the high dynamic range scene and the low dynamic range scene.

[0141] Step 1102: Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the second mapping intensity to determine a target image of the image to be processed.

[0142] Furthermore, in some embodiments, Fig.13 As shown, step 1102 includes the following steps 1301 to 1303:

[0143] Step 1301: Correct the pixel values ​​of the first segmented area in the first segmentation result to obtain a second segmented area.

[0144] It should be noted that the further implementation method of step 1301 is the same as the further implementation method of step 601 mentioned above. For the further implementation method of step 1301, please refer to the above description of the further implementation method of step 601, which will not be repeated here.

[0145] Step 1302: Determine a local mask corresponding to the segmented region according to the second segmented region and the first category label.

[0146] It should be noted that the further implementation method of step 1302 is the same as the further implementation method of step 602 mentioned above. For the further implementation method of step 1302, please refer to the above description of the further implementation method of step 602, which will not be repeated here.

[0147] Step 1303: Perform tone mapping processing on the image to be processed according to the first tone mapping curve, the second mapping intensity, and the local mask to determine a target image of the image to be processed.

[0148] In the embodiment of the present application, there is no limitation on the further implementation method of step 1303, that is, there is no limitation on how to use the first tone mapping curve, the second mapping intensity and the local mask to realize the tone mapping processing of the image to be processed, and there is no limitation on the order of operations based on the first tone mapping curve, the second mapping intensity and the local mask. In short, this information is required as input information for the tone mapping processing of the image to be processed.

[0149] Exemplarily, in some embodiments, step 1303 may further include: first mapping the pixel values ​​of the image to be processed using a first tone mapping curve to obtain a first mapping image, and then adjusting the pixel values ​​of the first mapping image using a second mapping intensity to obtain a third mapping image (for example, multiplying the second mapping intensity by the pixel value of the first mapping image to obtain the third mapping image); finally, determining the target image of the image to be processed based on the third mapping image and the local mask (multiplying the pixel value of each pixel position of the third mapping image with the value of the same pixel position in the local mask, and determining the target image based on this).

[0150] Exemplarily, in some other embodiments, step 1303 includes: determining a second tone mapping curve based on the first tone mapping curve and based on the second mapping intensity; and performing tone mapping processing on the image to be processed based on the second tone mapping curve and the local mask to determine a target image of the image to be processed.

[0151] It can be understood that the first tone mapping curve includes a grayscale or pixel value and a new pixel value corresponding to the grayscale or pixel value (i.e., the pixel value after tone mapping). For the above step of "determining the second tone mapping curve according to the first tone mapping curve and according to the second mapping intensity", in a possible implementation, the second mapping intensity can be applied to the pixel value in the first tone mapping curve, so as to adjust the pixel value in the first tone mapping curve, and the second tone mapping curve is determined based on this, for example, the second tone mapping curve is the adjusted first tone mapping curve, and of course, the second tone mapping curve can also be a curve obtained by further processing the adjusted first tone mapping curve, and this application does not limit this.

[0152] Here, for the further implementation method of the step of "performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine the target image of the image to be processed", please refer to the description of the further implementation method of step 1002 above, and the description will not be repeated here.

[0153] For step 104, tone mapping processing is performed on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

[0154] As mentioned above, in the embodiments of the present application, there is no limitation on the further implementation of step 104, which may be the scheme 1 and scheme 2 described above and the scheme 3, scheme 4 or scheme 5 described below.

[0155] Among them, in scheme 3, for step 104, further, in some other embodiments, as Fig.14 As shown, step 104 may include the following steps 1401 and 1403:

[0156] Step 1401, determining a first mapping strength according to the first category labels of the one or more first segmented regions;

[0157] Step 1402, determining a second mapping intensity according to a brightness change of the image to be processed;

[0158] Step 1403: Perform tone mapping processing on the image to be processed according to the first tone mapping curve, and according to the first mapping intensity and the second mapping intensity, to determine a target image of the image to be processed.

[0159] Understandably, in Fig.14 In the method for determining the target image shown, the tone mapping processing of the image to be processed is not only based on the first tone mapping curve, but also based on the first mapping intensity and the second mapping intensity. The first mapping intensity is determined according to the actual scene category (i.e., the first category label) of the one or more first segmented regions, and the second mapping intensity is determined according to the actual brightness change of the image to be processed; in this way, the tone mapping processing of the image to be processed is more in line with the characteristics of the image to be processed itself, which is beneficial to obtain a more realistic and natural target image with better tone levels.

[0160] The following describes further optional implementations and related terms of each of the above steps.

[0161] Step 1401: determine a first mapping strength according to the first category labels of the one or more first segmented regions.

[0162] It should be noted that the further implementation method of step 1401 is the same as the further implementation method of step 401 mentioned above. For the further implementation method of step 1401, please refer to the above description of the further implementation method of step 401, which will not be repeated here.

[0163] Step 1402: Determine a second mapping intensity according to a brightness change of the image to be processed.

[0164] It should be noted that the further implementation method of step 1402 is the same as the further implementation method of step 1101 mentioned above. For the further implementation method of step 1402, please refer to the above description of the further implementation method of step 1101, which will not be repeated here.

[0165] Step 1403: Perform tone mapping processing on the image to be processed according to the first tone mapping curve, and according to the first mapping intensity and the second mapping intensity, to determine a target image of the image to be processed.

[0166] Furthermore, in some embodiments, Fig.15 As shown, step 1403 includes the following steps 1501 to 1503:

[0167] Step 1501: Correct the pixel values ​​of the first segmented area in the first segmentation result to obtain a second segmented area.

[0168] It should be noted that the further implementation method of step 1501 is the same as the further implementation method of step 601 mentioned above. For the further implementation method of step 1501, please refer to the above description of the further implementation method of step 601, which will not be repeated here.

[0169] Step 1502: Determine a local mask corresponding to the segmented region according to the second segmented region and the first category label.

[0170] It should be noted that the further implementation method of step 1502 is the same as the further implementation method of step 602 mentioned above. For the further implementation method of step 1502, please refer to the above description of the further implementation method of step 602, which will not be repeated here.

[0171] Step 1503: Perform tone mapping processing on the image to be processed according to the first tone mapping curve, the first mapping intensity, the second mapping intensity, and the local mask to determine a target image of the image to be processed.

[0172] In the embodiments of the present application, there is no limitation on the further implementation manner of step 1503, that is, there is no limitation on how to use the first tone mapping curve, the first mapping intensity, the second mapping intensity, and the local mask to implement the tone mapping process of the image to be processed. There is no limitation on the operation order based on the first tone mapping curve, the first mapping intensity, the second mapping intensity, and the local mask. In short, these information are required as input information for the tone mapping process of the image to be processed.

[0173] Exemplarily, in some embodiments, step 1503 includes: determining a second tone mapping curve according to the first tone mapping curve, the first mapping intensity, and the second mapping intensity; and performing a tone mapping process on the image to be processed according to the second tone mapping curve and the local mask to determine the target image of the image to be processed.

[0174] It can be understood that the first tone mapping curve includes gray levels or pixel values and the corresponding new pixel values (i.e., the pixel values after tone mapping) of the gray levels or pixel values.

[0175] Among them, there is no limitation on the further implementation manner of the step of "determining a second tone mapping curve according to the first tone mapping curve, the first mapping intensity, and the second mapping intensity". In one implementation manner, a third mapping intensity may be determined according to the first mapping intensity and the second mapping intensity, and then the second tone mapping curve may be determined according to the third mapping intensity and the first tone mapping curve. For example, the third mapping intensity is the average value or weighted average value of the first mapping intensity and the second mapping intensity. In one possible implementation manner, the third mapping intensity may be applied to the pixel values in the first tone mapping curve, so as to adjust the pixel values in the first tone mapping curve, and based on this, the second tone mapping curve is determined. For example, the second tone mapping curve is the adjusted first tone mapping curve.

[0176] For the step of "determining a second tone mapping curve according to the first tone mapping curve, the first mapping intensity, and the second mapping intensity", in another implementation manner, a third tone mapping curve may also be determined according to the first mapping intensity and the first tone mapping curve; and a fourth tone mapping curve may be determined according to the second mapping intensity and the first tone mapping curve; the second tone mapping curve is determined according to the third tone mapping curve and the fourth tone mapping curve.

[0177] Among them, for determining the third tone mapping curve according to the first mapping intensity and the first tone mapping curve, for example, the first mapping intensity can be applied to the pixel values in the first tone mapping curve, so as to adjust the pixel values in the first tone mapping curve, and based on this, the third tone mapping curve is determined. For example, the third tone mapping curve is the adjusted first tone mapping curve.

[0178] Among them, for determining the fourth tone mapping curve according to the second mapping intensity and the first tone mapping curve, for example, the second mapping intensity can be applied to the pixel values in the first tone mapping curve, so as to adjust the pixel values in the first tone mapping curve, and based on this, the fourth tone mapping curve is determined. For example, the fourth tone mapping curve is the adjusted first tone mapping curve;

[0179] Among them, for determining the second tone mapping curve according to the third tone mapping curve and the fourth tone mapping curve, for example, the new pixel values corresponding to the gray levels or pixel values of the third tone mapping curve can be averaged or weighted averaged with the new pixel values at the corresponding gray levels or pixel values of the fourth tone mapping curve, and based on this, the second tone mapping curve is determined (for example, the tone mapping curve obtained by this averaging or weighted averaging is the second tone mapping curve).

[0180] For the further implementation of the step of "performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine the target image of the image to be processed", reference can be made to the description of the further implementation of step 1002 above for understanding, and details are not described here again.

[0181] It can be understood that for the mapping intensity considered in the above-mentioned solutions 1 to 3, the first mapping intensity and / or the second mapping intensity participate in the tone mapping processing of the image to be processed. Of course, the first mapping intensity and the second mapping intensity may also not participate in the tone mapping processing of the image to be processed.

[0182] That is, for step 104, according to the first tone mapping curve, perform tone mapping processing on the image to be processed to determine the target image of the image to be processed.

[0183] In Solution 4, in some embodiments, as Fig.16 shown, step 104 may include the following steps 1601 to 1603:

[0184] Step 1601, correct the pixel values of the first segmentation region in the first segmentation result to obtain a second segmentation region.

[0185] It should be noted that the further implementation method of step 1601 is the same as the further implementation method of step 601 mentioned above. For the further implementation method of step 1601, please refer to the above description of the further implementation method of step 601, which will not be repeated here.

[0186] Step 1602: Determine a local mask corresponding to the segmented region according to the second segmented region and the first category label.

[0187] It should be noted that the further implementation method of step 1602 is the same as the further implementation method of step 602 mentioned above. For the further implementation method of step 1602, please refer to the above description of the further implementation method of step 602, which will not be repeated here.

[0188] Step 1603: Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the local mask to determine a target image of the image to be processed.

[0189] In a possible implementation, the result obtained by the tone mapping process can be directly used as the target image. For example, the target image of the image to be processed can be determined by the following formula:

[0190] Target image = Global[input]*local mask+(1-local mask)*input;

[0191] Among them, Global[] refers to the first tone mapping curve, input refers to the image to be processed, and local mask refers to a local mask image composed of all local masks corresponding to the image to be processed.

[0192] In another possible implementation, the result obtained by the tone mapping (which we call an intermediate image) can be further processed (such as boundary fusion processing) to obtain the target image; that is, step 1603 further includes: according to the first tone mapping curve and the local mask, tone mapping processing is performed on the image to be processed to obtain an intermediate image; and the boundary area of ​​the second segmented area in the intermediate image is fused to determine the target image.

[0193] For example, the intermediate image can be determined by the following formula:

[0194] Intermediate image = Global[input]*local mask+(1-local mask)*input;

[0195] Among them, Global[] refers to the first tone mapping curve, input refers to the image to be processed, and local mask refers to the local mask image composed of all local masks corresponding to the image to be processed.

[0196] For step 104, perform tone mapping processing on the image to be processed according to the first tone mapping curve to determine the target image of the image to be processed.

[0197] As mentioned above, in the embodiments of the present application, there is no limitation on the further implementation manner of step 104. It can be any one of the above-mentioned solutions 1 to 4, or solution 5 described below, etc.

[0198] Among them, in solution 5, for step 104, in some embodiments, it is possible not to correct the pixel values of the first segmentation region, but to determine the local mask of the corresponding segmentation region based on the first segmentation region and the first category label; and perform tone mapping processing on the image to be processed according to the first tone mapping curve, and according to the first mapping intensity and / or the second mapping intensity, and according to the local mask, to determine the target image of the image to be processed.

[0199] Among other solutions, for step 104, further, in some embodiments, it is also possible not to calculate the local masks corresponding to each segmentation region respectively, but directly use the first tone mapping curve to perform tone mapping processing on the image to be processed to determine the target image of the image to be processed.

[0200] In a possible implementation manner, the result obtained by the tone mapping processing can be directly used as the target image. For example, the target image of the image to be processed can be determined by the following formula:

[0201] Target image = Global[input];

[0202] Among them, Global[] refers to the first tone mapping curve, and input refers to the image to be processed.

[0203] In another possible implementation manner, the result obtained by the tone mapping (which we call the intermediate image) can be further processed (such as boundary fusion processing) to obtain the target image; that is, it can further include: performing tone mapping processing on the image to be processed according to the first tone mapping curve to obtain an intermediate image; performing fusion processing on the boundary region of the first segmentation region in the intermediate image to determine the target image.

[0204] For example, the intermediate image can be determined by the following formula:

[0205] middleImage = Global[input];

[0206] Wherein, Global[] refers to the first tone mapping curve, and input refers to the image to be processed.

[0207] In summary, there is no limitation on the further implementation of step 104. In summary, the input information involved in the tone mapping process of the image to be processed at least includes the first tone mapping curve.

[0208] The following examples describe possible implementations of the image processing methods described in one or more of the above embodiments.

[0209] Users' attention to video effects has also gradually shifted from basic aspects such as high definition and image stability to stylized characteristics such as capturing a more natural and realistic world. The most important aspect is the tone effect of the video.

[0210] Tone refers to the light and dark levels of a video image, which is mainly used to create atmosphere, express emotions and shape characters. According to the different hues and brightness, tones are divided into highlights, shadows and midtones. Among them, highlights are the areas with the highest brightness in an image or picture, midtones refer to areas in an image or picture that are neither bright nor dark, and shadows refer to darker areas in an image or picture.

[0211] Dolby Vision came into being, and its main features include high dynamic range (HDR) and wide color gamut, making the picture more vivid and realistic. Dolby Vision can retain more details in both bright and dark areas, present richer contrast and more accurate color performance, allowing users to experience a more immersive visual effect. In addition, Dolby Vision supports dynamic metadata, which adjusts the brightness, contrast and color according to the specific content of each frame to ensure the best effect on different display devices, and also makes Dolby Vision have good consistency in effect between different communication media.

[0212] The following embodiment provides a method for enhancing the tone of a video combined with an AI segmentation network (that is, an exemplary implementation of the above-mentioned image processing method). This method is suitable for enhancing the tone of Dolby video, and aims to further improve the real-life shooting effect of Dolby video, and obtain a more realistic, natural, and comfortable tone level. Different from other contrast stretching methods that rely on tone mapping, the following scheme combines an image segmentation network to achieve scene discrimination and generate a tone mapping curve for the content of the picture. At the same time, combined with the results of image segmentation, different degrees of local enhancement can be achieved for different segmented areas.

[0213] In the following scheme, scene discrimination is performed in combination with the segmentation results, and a global mapping of the corresponding scene is given to different scenes; after obtaining the weight factors of each segmented area, the weights of each category are fused and then combined with the global mapping for application; and the segmentation map (i.e., the segmentation map) is transformed into a weight map (i.e., the local mask), and boundary fusion is performed to ensure the intensity control of the local area.

[0214] The usage method based on the segmentation result in the embodiment of the present application is different from the related technology: in the related technology, the tone mapping information is obtained based on the correspondence between the segmented area and at least one related metadata information unit, and this correspondence is predefined, that is, the tone mapping information is predefined; while in the embodiment of the present application, the first tone mapping curve is determined based on the scene category and pixel value distribution of the segmented area, which are information reflecting the actual image content; therefore, compared with the related technology, the tone mapping curve obtained in the embodiment of the present application is more in line with the actual content of the image to be processed, so the final target image is more realistic, natural, and has a better tone level.

[0215] That is to say, the relevant video tone enhancement technology does not combine the actual content of the image to distinguish the style of the scene, and the mapping method of the whole scene is single. The inventors of this application found in research and analysis that if the mapping method of the whole scene is single, then in the process of achieving different intensities in different segmented areas, tone flipping is very likely to occur, and intensity performance opposite to the real world will appear (such as the brightness of the reflective ground is higher than the bright sky); when the intensities in different areas are different, problems such as halo and local flicker will occur.

[0216] In view of this, a video tone enhancement algorithm solution combined with an AI segmentation network is described below (that is, an exemplary implementation solution of the above-mentioned image processing method). Using the results of the AI ​​segmentation network, combined with information such as the image histogram, the category of the scene is first determined, and the tone mapping curve (global tonecurve) belonging to the scene is obtained. For different segmented areas, a local mask is generated for the area of ​​interest (such as the sky, buildings, etc.), and local tone mapping (local LCE) is implemented for the segmented area. Combined with the segmentation results (seg map), the boundaries of different segmented areas are fused to achieve a natural transition of the boundaries to eliminate local halo; in the field of photography and image processing, halo refers to the edge halo effect caused by over-sharpening or highlight overflow in the image. The following describes the video tone enhancement algorithm solution combined with the AI ​​segmentation network.

[0217] 1. Calculate global tone

[0218] 1.1 Scene determination using segmentation results: obtaining global mapping

[0219] The algorithm first combines the segmentation map and image content information of the image to be processed to identify the current scene. Figure 2 shown.

[0220] Each segmented area has a corresponding segmentation label (i.e., the first category label), and the main label of the current scene can be determined based on the segmentation label. Whether it is a portrait scene is the first-level judgment condition, and the area ratio is the second judgment condition (the top two with the largest ratio are retained);

[0221] like:

[0222] The segmentation label of segmentation area 1 (i.e., the first category label) has a sky probability of 95+%, so the identified scene category is sky;

[0223] The segmentation labels of segmentation areas 2 and 5 (i.e., the first category labels) have a probability of 50% for green plants and 50% for buildings. Therefore, the segmentation label of segmentation area 2 (i.e., the first category label) identifies the scene category as "green plants";

[0224] The scene category identified by the segmentation label (i.e., the first category label) of segmentation area 5 is “building”;

[0225] The segmentation label of segmentation area 3 (i.e., the first category label) has a probability of 80% for human. Therefore, the segmentation label of segmentation area 3 (i.e., the first category label) identifies the scene category as “human”.

[0226] The segmentation label of segmentation area 4 (ie, the first category label) has a street probability of 80%, so the scene category identified by the segmentation label of segmentation area 4 (ie, the first category label) is “street”.

[0227] The scene labels are:

[0228] Portrait-Sky / Street / Greenery / Architecture / Others;

[0229] Or, non-portraits - sky / streets / greenery / buildings / others;

[0230] Different first mapping intensities are distinguished according to whether a portrait is included, that is, the first mapping intensities corresponding to the portrait and the non-portrait are different;

[0231] For different types of environments such as the sky and buildings, global tone presets different curves. The calculation method is as follows: take the histogram statistics corresponding to the first two scene categories (first category labels) with the largest area share, generate a first cumulative histogram, and determine the first tone mapping curve based on the first cumulative histogram; determine the corresponding first mapping intensity in combination with the main label;

[0232] 1.2 Scene determination using image histograms: obtaining global map strength

[0233] Combined with the scene content information, the current scene is determined based on the image's ambient brightness and histogram. The method of determining the environment from the histogram is:

[0234] Count the proportion of different brightness ranges in the whole image:

[0235] like Fig.12 As shown in the figure, if 128 is used as the boundary, and the left and right sides have a larger proportion (such as the proportion of pixels with brightness values ​​less than 128 is greater than 40%, and the proportion of pixels with brightness values ​​greater than 128 is greater than 40%), the image is considered to be a high dynamic range scene;

[0236] like Fig.12 As shown, the image has a large proportion near 128 (e.g., the proportion of pixels with brightness values ​​between 110 and 150 is >50%), which is considered a low dynamic range scene;

[0237] like Fig.12 As shown, the image contains multiple peaks and the histogram is dispersed, which is considered to be a mixed scene; for example, if it is not a high dynamic range scene or a low dynamic range scene, then it is determined to be a mixed scene.

[0238] The judgment result of the histogram indexes different effective intensity values. In the case of high dynamic range, the effective intensity is the largest, and the effective intensity is the smallest in the mixed scene.

[0239] 2. Generate local mask

[0240] The local mask is obtained by calculating the segmentation map, which indicates the local effective intensity. For example, if you do not want to do contrast stretching in the dark area to avoid the dead black problem, you can do local protection / partial protection for buildings and green plants in the dark area.

[0241] First, the segmentation map is checked for accuracy, because there may be local misdetection in the original segmentation result (such as Figure 5 As shown in the figure, using the segmentation result of the wrong detection will cause the problem of local brightness blocks.

[0242] The verification steps are as follows:

[0243] (a) converting the input data (i.e., the image to be processed) into a first grayscale image, resizing it to the size of the segmentation image, and obtaining a second grayscale image;

[0244] (b) performing edge detection on the second grayscale image to obtain edge information (i.e., edge map);

[0245] (c) subtracting the segmentation result / segmentation region i from the edge map, and marking pixels whose difference is greater than or equal to a second threshold;

[0246] (d) In the edge map, calculate the coordinate position (i.e., similar point) with similar grayscale to the current position, replace the segmentation result of the marked pixel point with the weighted sum of the grayscale values ​​of the similar point, and obtain the verified segmentation result;

[0247] After obtaining the verified segmentation result, the category to be protected (the sub-label corresponding to the segmented area) is set to 0; according to the first category label, the verified segmentation result is multiplied by 1 corresponding adjustment coefficient to obtain the third segmented area, and the segmentation map of the third segmented area is resized to the original image size, and then normalized to obtain the local mask.

[0248] Output result = Global[input]*local mask+(1-local mask)*input

[0249] 3. Boundary Area Fusion

[0250] At the boundary of the segmentation, for the pixel points at the boundary of the region in the output result of step 2, calculate the shortest distance from the current point to each adjacent region, convert the distance into a similarity weight (the larger the distance, the smaller the weight), and the tone curve result of the current point is:

[0251] Lut(x,y)=region1(x,y)*weight1+region2(x,y)*weight2+…

[0252] Wherein, Lut(x,y) is an example of the target image, region1(x,y): the boundary value of the output result of step 2. Fig.17 The weight diagram of the segmentation map and the corresponding segmentation boundary area provided in the embodiment of the present application is shown in FIG1701 , and the weight diagram of the corresponding segmentation boundary area is shown in FIG1702 .

[0253] It can be understood that in the embodiment of the present application, a tone enhancement algorithm scheme combined with an AI segmentation network is described. The scheme can realize image scene determination by combining image information with the segmentation result, thereby obtaining a global mapping curve and its strength. At the same time, different weight controls can be implemented for different segmentation areas. The image can be more accurately enhanced in tone.

[0254] Experimental verification shows that the above algorithm is an effective and highly accurate tone enhancement algorithm.

[0255] It can be understood that in the above-mentioned tone enhancement algorithm applied to video streams, (1) the scene is judged by combining image information and segmentation results. Different scenes are combined with different glocal weights and given different intensity controls, which can achieve the best global tone relationship without tone flipping; (2) the accuracy of the segmentation results is checked, and local intensity control can be achieved for different segmented areas, which is very flexible;

[0256] It should be noted that the embodiments of the present application can be used for input images with any bit width (bit), such as 8-bit corresponding data range (0-255) and 10-bit data range (0-1024). The scheme makes a distinction based on the input format and can all be successfully processed.

[0257] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.; or, steps in different embodiments may be combined into a new technical solution.

[0258] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which includes the modules included and the units included in the modules, which can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an AI acceleration engine (such as NPU, etc.), ISP, GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0259] Fig.18 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application; Fig.18 As shown, the image processing device 180 includes an acquisition module 1801, a first determination module 1802, a second determination module 1803 and a third determination module 1804, wherein:

[0260] The acquisition module 1801 is configured to: acquire a first segmentation result of the image to be processed, wherein the first segmentation result includes one or more first segmented regions and a first category label of the first segmented region, wherein the first category label is used to identify a scene category of the corresponding segmented region;

[0261] A first determination module 1802 is configured to: select one or more target segmented regions from the one or more first segmented regions according to a first category label of the first segmented region;

[0262] The second determination module 1803 is configured to: determine a first tone mapping curve according to the pixel value distribution of the one or more target segmented areas;

[0263] The third determination module 1804 is configured to: perform tone mapping processing on the image to be processed according to the first tone mapping curve, and determine a target image of the image to be processed.

[0264] In some embodiments, selecting one or more target segmented regions from the one or more first segmented regions according to the first category label of the first segmented region includes:

[0265] Determining a main scene category of the image to be processed according to a first category label of the first segmented area;

[0266] Determine a first segmented area corresponding to a non-main scene category from the one or more first segmented areas;

[0267] One or more target segmented regions are selected from the first segmented regions corresponding to the non-main scene categories.

[0268] Furthermore, in some embodiments, the selecting one or more target segmented areas from the first segmented areas corresponding to the non-main scene categories includes:

[0269] Determine size-related parameters of the first segmented area corresponding to the non-main scene category;

[0270] One or more target segmentation areas whose size-related parameters satisfy a first condition are selected from the first segmentation areas corresponding to the non-main scene category; wherein the first condition includes that the size-related parameters are greater than or equal to a first threshold, or the first condition includes the first K maximum values, K being greater than or equal to 1.

[0271] In some embodiments, determining the first tone mapping curve according to the pixel value distribution of the one or more target segmented areas includes:

[0272] Performing statistics on pixel value distribution of the one or more target segmented regions to determine a first cumulative histogram of the one or more target segmented regions;

[0273] The first tone mapping curve is determined according to the first cumulative histogram.

[0274] In some embodiments, performing tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed includes:

[0275] Determine a first mapping strength according to the first category labels of the one or more first segmented regions, and / or determine a second mapping strength according to a brightness change of the image to be processed;

[0276] According to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity, tone mapping processing is performed on the image to be processed to determine a target image of the image to be processed.

[0277] Furthermore, in some embodiments, determining the first mapping strength according to the first category labels of the one or more first segmented regions includes:

[0278] Determining a main scene category of the image to be processed according to the first category labels of the one or more first segmented regions;

[0279] According to the main scene category, a corresponding first mapping intensity is determined.

[0280] In some embodiments, the brightness change of the image to be processed includes a high dynamic range scene, a low dynamic range scene, or a mixed scene of the high dynamic range scene and the low dynamic range scene;

[0281] Among them, the second mapping intensity corresponding to the high dynamic range scene is greater than the second mapping intensity corresponding to the mixed scene, and the second mapping intensity corresponding to the mixed scene is greater than the second mapping intensity corresponding to the low dynamic range scene.

[0282] Further, in some embodiments, performing tone mapping processing on the image to be processed according to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity to determine the target image of the image to be processed includes:

[0283] Correcting pixel values ​​of a first segmented area in the first segmentation result to obtain a second segmented area;

[0284] Determining a local mask corresponding to the segmented area according to the second segmented area and the first category label;

[0285] According to the first tone mapping curve, according to the first mapping intensity and / or the second mapping intensity, and according to the local mask, tone mapping processing is performed on the image to be processed to determine a target image of the image to be processed.

[0286] In some other embodiments, performing tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed includes:

[0287] Correcting pixel values ​​of a first segmented area in the first segmentation result to obtain a second segmented area;

[0288] Determining a local mask corresponding to the segmented area according to the second segmented area and the first category label;

[0289] Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the local mask to determine a target image of the image to be processed.

[0290] Furthermore, in some embodiments, correcting the pixel values ​​of the first segmented area in the first segmentation result to obtain the second segmented area includes:

[0291] Determine a first grayscale image of the image to be processed;

[0292] Scaling the first grayscale image to a segmentation image size corresponding to the first segmentation result to obtain a second grayscale image;

[0293] Performing edge detection on the second grayscale image to determine edge pixels of the second grayscale image;

[0294] Determine the pixel to be corrected in the first segmented area according to the pixel value of the edge pixel in the second grayscale image; the difference between the pixel value of the pixel to be corrected and the pixel value of the co-located edge pixel is greater than or equal to a second threshold;

[0295] Correct the pixels to be corrected in the first segmented area to obtain a second segmented area.

[0296] Exemplarily, in some embodiments, correcting the pixels to be corrected in the first segmented area to obtain the second segmented area includes:

[0297] Searching for a reference pixel having a pixel value similar to that of the pixel to be corrected from the second grayscale image;

[0298] The pixel value of the corresponding pixel to be corrected is corrected according to the pixel value of the reference pixel to obtain a second segmented area.

[0299] In a possible implementation, the correcting the pixel value of the corresponding pixel to be corrected according to the pixel value of the reference pixel to obtain the second segmented area includes:

[0300] determining an average or a weighted average of the pixel values ​​of the reference pixels;

[0301] The pixel values ​​corresponding to the pixels to be corrected are corrected according to the average value or the weighted average value to obtain a second segmented area.

[0302] Furthermore, in some embodiments, determining a local mask corresponding to the segmented region according to the second segmented region and the first category label includes:

[0303] Determining, according to the first category label, an adjustment coefficient of a pixel value corresponding to the segmented area;

[0304] Adjust the pixel value of the corresponding second segmented area according to the adjustment coefficient to obtain a third segmented area;

[0305] A corresponding local mask is determined according to the third segmented area.

[0306] Exemplarily, in some embodiments, determining a corresponding local mask according to the third segmented area includes:

[0307] Converting a first segmentation map composed of third segmentation areas corresponding to the one or more first segmentation areas to a size of the image to be processed to obtain a second segmentation map;

[0308] Normalizing the pixel values ​​of the second segmentation map to obtain a local mask corresponding to the third segmentation area.

[0309] In some embodiments, performing tone mapping processing on the image to be processed according to the first tone mapping curve, according to the first mapping intensity and / or the second mapping intensity, and according to the local mask to determine the target image of the image to be processed includes:

[0310] Determining a second tone mapping curve according to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity;

[0311] Perform tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine a target image of the image to be processed.

[0312] Furthermore, in some embodiments, performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine a target image of the image to be processed includes:

[0313] Performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to obtain an intermediate image;

[0314] A fusion process is performed on the boundary area of ​​the second segmented area in the intermediate image to determine the target image.

[0315] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the above method embodiment for understanding.

[0316] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0317] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0318] An embodiment of the present application provides an electronic device, Fig.19 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application; Fig.19 As shown, the electronic device 190 includes a memory 1901 and a processor 1902, wherein the memory 1901 stores a computer program that can be run on the processor 1902, and the processor 1902 implements the steps in the method provided in the above embodiment when executing the program.

[0319] It should be noted that the memory 1901 is configured to store instructions and applications executable by the processor 1902, and can also cache data to be processed or already processed by the processor 1902 and various modules in the electronic device 190 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0320] In the embodiments of the present application, there is no limitation on the type of electronic device, and the electronic device may be any device with image processing capability, such as a mobile phone, a laptop computer, a tablet computer, a smart home device, or a vehicle-mounted device.

[0321] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0322] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and the computer program enables the processor or electronic device to execute the various methods of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0323] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0324] Optionally, the computer program product may be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the processor or electronic device to execute the various methods in the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0325] The embodiment of the present application also provides a computer program.

[0326] Optionally, the computer program may be applied to the electronic device in the embodiments of the present application. When the computer program runs on a processor or an electronic device, the processor or the electronic device executes the various methods in the embodiments of the present application. For the sake of brevity, they are not described here in detail.

[0327] It should be noted here that the description of the above electronic device, storage medium, computer program product and computer program embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device, storage medium, computer program product and computer program embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0328] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0329] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0330] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0331] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0332] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0333] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0334] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0335] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0336] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain a new method embodiment. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain a new product embodiment. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain a new method embodiment or device embodiment.

[0337] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first segmentation result of the image to be processed, where the first segmentation result includes one or more first segmented regions and first category labels of the first segmented regions, where the first category labels are used to identify a scene category of the corresponding segmented regions; Selecting one or more target segmented regions from the one or more first segmented regions according to the first category label of the first segmented region; Determining a first tone mapping curve according to a distribution of pixel values ​​of the one or more target segmented areas; According to the first tone mapping curve, tone mapping processing is performed on the image to be processed to determine a target image of the image to be processed.

2. The method according to claim 1, characterized in that The selecting one or more target segmented regions from the one or more first segmented regions according to the first category label of the first segmented region comprises: Determining a main scene category of the image to be processed according to a first category label of the first segmented area; Determine a first segmented area corresponding to a non-main scene category from the one or more first segmented areas; One or more target segmented regions are selected from the first segmented regions corresponding to the non-main scene categories.

3. The method according to claim 2, characterized in that The step of selecting one or more target segmentation areas from the first segmentation areas corresponding to the non-main scene categories includes: Determine size-related parameters of the first segmented area corresponding to the non-main scene category; One or more target segmentation areas whose size-related parameters satisfy a first condition are selected from the first segmentation areas corresponding to the non-main scene category; wherein the first condition includes that the size-related parameters are greater than or equal to a first threshold, or the first condition includes the first K maximum values, K being greater than or equal to 1.

4. The method according to claim 1, characterized in that The determining of a first tone mapping curve according to the pixel value distribution of the one or more target segmented areas includes: Performing statistics on pixel value distribution of the one or more target segmented regions to determine a first cumulative histogram of the one or more target segmented regions; The first tone mapping curve is determined according to the first cumulative histogram.

5. The method according to any one of claims 1 to 4, characterized in that The step of performing tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed includes: Determine a first mapping strength according to the first category labels of the one or more first segmented regions, and / or determine a second mapping strength according to a brightness change of the image to be processed; According to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity, tone mapping processing is performed on the image to be processed to determine a target image of the image to be processed.

6. The method according to claim 5, characterized in that The determining the first mapping strength according to the first category labels of the one or more first segmented regions comprises: Determining a main scene category of the image to be processed according to the first category labels of the one or more first segmented regions; According to the main scene category, a corresponding first mapping intensity is determined.

7. The method according to claim 5, characterized in that The brightness change of the image to be processed includes a high dynamic range scene, a low dynamic range scene, or a mixed scene of the high dynamic range scene and the low dynamic range scene; Among them, the second mapping intensity corresponding to the high dynamic range scene is greater than the second mapping intensity corresponding to the mixed scene, and the second mapping intensity corresponding to the mixed scene is greater than the second mapping intensity corresponding to the low dynamic range scene.

8. The method according to claim 5, characterized in that The step of performing tone mapping processing on the image to be processed according to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity to determine a target image of the image to be processed includes: Correcting pixel values ​​of a first segmented area in the first segmentation result to obtain a second segmented area; Determining a local mask corresponding to the segmented area according to the second segmented area and the first category label; According to the first tone mapping curve, according to the first mapping intensity and / or the second mapping intensity, and according to the local mask, tone mapping processing is performed on the image to be processed to determine a target image of the image to be processed.

9. The method according to any one of claims 1 to 4, characterized in that: The step of performing tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed includes: Correcting pixel values ​​of a first segmented area in the first segmentation result to obtain a second segmented area; Determining a local mask corresponding to the segmented area according to the second segmented area and the first category label; Perform tone mapping processing on the image to be processed according to the first tone mapping curve and the local mask to determine a target image of the image to be processed.

10. The method according to claim 8 or 9, characterized in that: The step of correcting the pixel values ​​of the first segmented area in the first segmentation result to obtain the second segmented area includes: Determine a first grayscale image of the image to be processed; Scaling the first grayscale image to a segmentation image size corresponding to the first segmentation result to obtain a second grayscale image; Performing edge detection on the second grayscale image to determine edge pixels of the second grayscale image; Determine the pixel to be corrected in the first segmented area according to the pixel value of the edge pixel in the second grayscale image; the difference between the pixel value of the pixel to be corrected and the pixel value of the co-located edge pixel is greater than or equal to a second threshold; Correct the pixels to be corrected in the first segmented area to obtain a second segmented area.

11. The method according to claim 10, characterized in that The correcting the pixels to be corrected in the first segmented area to obtain the second segmented area includes: Searching for a reference pixel having a pixel value similar to that of the pixel to be corrected from the second grayscale image; The pixel value of the corresponding pixel to be corrected is corrected according to the pixel value of the reference pixel to obtain a second segmented area.

12. The method according to claim 11, characterized in that The step of correcting the pixel value of the corresponding pixel to be corrected according to the pixel value of the reference pixel to obtain a second segmented area comprises: determining an average or a weighted average of the pixel values ​​of the reference pixels; The pixel values ​​corresponding to the pixels to be corrected are corrected according to the average value or the weighted average value to obtain a second segmented area.

13. The method according to claim 8 or 9, characterized in that: The determining of a local mask corresponding to the segmented area according to the second segmented area and the first category label includes: Determining, according to the first category label, an adjustment coefficient of a pixel value corresponding to the segmented area; Adjust the pixel value of the corresponding second segmented area according to the adjustment coefficient to obtain a third segmented area; A corresponding local mask is determined according to the third segmented area.

14. The method according to claim 13, characterized in that The determining a corresponding local mask according to the third segmented area includes: Converting a first segmentation map composed of third segmentation areas corresponding to the one or more first segmentation areas to a size of the image to be processed to obtain a second segmentation map; Normalizing the pixel values ​​of the second segmentation map to obtain a local mask corresponding to the third segmentation area.

15. The method according to claim 8, characterized in that The step of performing tone mapping processing on the image to be processed according to the first tone mapping curve, the first mapping intensity and / or the second mapping intensity, and the local mask to determine a target image of the image to be processed includes: Determining a second tone mapping curve according to the first tone mapping curve and according to the first mapping intensity and / or the second mapping intensity; Perform tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine a target image of the image to be processed.

16. The method according to claim 15, characterized in that The step of performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to determine a target image of the image to be processed includes: Performing tone mapping processing on the image to be processed according to the second tone mapping curve and the local mask to obtain an intermediate image; A fusion process is performed on the boundary area of ​​the second segmented area in the intermediate image to determine the target image.

17. An image processing device, characterized in that: The device comprises: an acquisition module configured to acquire a first segmentation result of the image to be processed, wherein the first segmentation result includes one or more first segmented regions and a first category label of the first segmented region, wherein the first category label is used to identify a scene category of the corresponding segmented region; A first determination module, configured to select one or more target segmented regions from the one or more first segmented regions according to a first category label of the first segmented region; a second determination module configured to determine a first tone mapping curve according to a distribution of pixel values ​​of the one or more target segmented regions; The third determination module is configured to perform tone mapping processing on the image to be processed according to the first tone mapping curve to determine a target image of the image to be processed.

18. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 16 is implemented.

19. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the method according to any one of claims 1 to 16 is implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 16 is implemented.

Images