Color evaluation-based image color enhancement method and apparatus, and device

Abstract

Embodiments of the present application provide a color evaluation-based image color enhancement method and apparatus, and a device. The method comprises: acquiring an input first image, performing image segmentation processing on the first image, to obtain multiple segmentation regions, and performing color space conversion processing on the first image, to obtain a second image of a preset color space mode; based on a component value of a color space corresponding to a segmentation region in the second image, calculating a regional color evaluation value of each segmentation region; by means of a configured intensity prediction module, determining a color adjustment coefficient corresponding to the regional color evaluation value; based on the color adjustment coefficient, performing color enhancement on the first image. The image color enhancement method can adapt to complex and changeable live broadcast scenarios, the complexity of color enhancement processing is low, and the color enhancement effect is improved.

Description

Image color enhancement method, device and equipment based on color evaluation

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 29, 2023, with application number 202311863106.X, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technology, and in particular to a method, device, and apparatus for image color enhancement based on color evaluation. Background Art

[0003] With the rise of live streaming, optimizing live streaming technology and improving video perception have become major research topics. Most standard live streamers experience issues like dim colors, gray fog, and unclear video due to camera capture and video compression. When these source streams undergo server-side transcoding and reach viewers, they experience a poor viewing experience, characterized by unclear video and dull colors. Therefore, color enhancement of captured video images to enhance the subjective experience of the video has become a necessary technology in the live streaming field.

[0004] Related technologies often use adaptive mean square equalization (AMSE) to enhance image color, or employ end-to-end generative models to directly generate the enhanced image from the input image. These AMSE-based approaches are inadequate for live broadcasts with complex and changing image content. Using an end-to-end model to generate the enhanced image incurs high computational overhead and complexity, and the resulting processing results are unpredictable. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, and device for image color enhancement based on color evaluation. After segmenting the input image to obtain multiple segmented regions, the regional color evaluation value is calculated based on the component values ​​of the color space of the segmented regions. After determining the color adjustment coefficient using the regional color evaluation value, the color of the image is enhanced. This color enhancement method can adapt to complex and changeable live broadcast scenes, has low complexity in color enhancement processing, and has better color enhancement effect.

[0006] In a first aspect, an embodiment of the present application provides an image color enhancement method based on color evaluation, the method comprising:

[0007] Acquire an input first image, perform image segmentation processing on the first image to obtain a plurality of segmented regions, and perform color space conversion processing on the first image to obtain a second image in a preset color space mode;

[0008] Calculating a regional color evaluation value of each of the segmented regions based on component values ​​of the color space corresponding to the segmented regions in the second image;

[0009] Determine the color adjustment coefficient corresponding to the color evaluation value of the region by using a set intensity prediction module;

[0010] Color enhancement is performed on the first image based on the color adjustment coefficient.

[0011] In a second aspect, an embodiment of the present application further provides an image color enhancement device based on color evaluation, the device comprising:

[0012] An acquisition module configured to acquire an input first image;

[0013] an image segmentation module configured to perform image segmentation processing on the first image to obtain a plurality of segmented regions;

[0014] An image conversion module configured to perform color space conversion processing on the first image to obtain a second image in a preset color space mode;

[0015] an evaluation value calculation module configured to calculate a regional color evaluation value of each of the segmented regions based on component values ​​of the color space corresponding to the segmented regions in the second image;

[0016] an intensity prediction module configured to determine a color adjustment coefficient corresponding to the color evaluation value of the region;

[0017] A color adjustment module is configured to perform color enhancement on the first image based on the color adjustment coefficient.

[0018] In a third aspect, an embodiment of the present application further provides an image color enhancement device based on color evaluation, the device comprising:

[0019] one or more processors;

[0020] a storage device configured to store one or more programs,

[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the image color enhancement method based on color evaluation described in the embodiment of the present application.

[0022] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are configured to execute the image color enhancement method based on color evaluation described in an embodiment of the present application.

[0023] In a fifth aspect, an embodiment of the present application further provides a program for image color enhancement based on color evaluation. When the program is executed, operations related to the image color enhancement method based on color evaluation as described in the first aspect can be implemented.

[0024] In an embodiment of the present application, a first image is obtained as an input, the first image is segmented to obtain a plurality of segmented regions, and the first image is converted to a color space to obtain a second image in a preset color space mode. Based on the component values ​​of the color space corresponding to the segmented regions in the second image, a regional color evaluation value of each segmented region is calculated, a color adjustment coefficient corresponding to the regional color evaluation value is determined by a set intensity prediction module, and the first image is color enhanced based on the color adjustment coefficient. In the above scheme, the image is segmented to obtain a plurality of segmented regions, and the color component values ​​after color space conversion are used to perform color evaluation on each segmented region. This scheme can adapt to scenes with complex image content. After the color adjustment coefficient is determined by the intensity prediction module, the color adjustment coefficient is used to achieve color enhancement of the original image. Since the color adjustment coefficient is determined by the regional color evaluation value, and in the process of determining the regional color evaluation value, different segmented regions correspond to different regional color evaluation values, the image enhancement effect after the color enhancement is better. At the same time, the enhanced image of the original image is not directly input using an end-to-end generation model, which reduces the computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a flow chart of an image color enhancement method based on color evaluation provided by an embodiment of the present application;

[0026] FIG2 is a flowchart of another method for image color enhancement based on color evaluation provided by an embodiment of the present application;

[0027] FIG3 is a schematic diagram of semantic segmentation of an image provided by an embodiment of the present application;

[0028] FIG4 is a flowchart of another method for image color enhancement based on color evaluation provided by an embodiment of the present application;

[0029] FIG5 is a flowchart of another method for image color enhancement based on color evaluation provided by an embodiment of the present application;

[0030] FIG6 is a schematic diagram of foreground and background segmentation processing provided by an embodiment of the present application;

[0031] FIG7 is a flowchart of another method for image color enhancement based on color evaluation provided by an embodiment of the present application;

[0032] FIG8 is a schematic diagram of a saturation coefficient mask provided in an embodiment of the present application;

[0033] FIG9 is a schematic diagram of a framework of an image color enhancement method based on color evaluation provided in an embodiment of the present application;

[0034] FIG10 is a structural block diagram of an image color enhancement device based on color evaluation provided by an embodiment of the present application;

[0035] FIG11 is a schematic structural diagram of an image color enhancement device based on color evaluation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the embodiments of the present application, rather than all structures.

[0037] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0038] The color enhancement method based on color assessment provided in the embodiments of this application can enhance the color of an input image, which can be a single image or a frame from a captured video stream. This method can be applied to scenarios such as live broadcasts, video conferencing, and video calls. The aforementioned application scenarios are merely exemplary and illustrative; in actual applications, color enhancement can also be performed on images captured in other scenarios.

[0039] In the image color enhancement method based on color evaluation provided in the embodiment of the present application, the execution entity of each step can be a computer device, which refers to any electronic device with data calculation, processing and storage capabilities, such as mobile phones, PCs (Personal Computers), tablet computers and other terminal devices, and can also be servers and other devices. The embodiment of the present application does not limit this.

[0040] FIG1 is a flow chart of a method for enhancing image color based on color evaluation provided by an embodiment of the present application. As shown in FIG1 , the method includes the following steps:

[0041] Step S101: Acquire an input first image, perform image segmentation processing on the first image to obtain a plurality of segmented regions, and perform color space conversion processing on the first image to obtain a second image in a preset color space mode.

[0042] Among them, the first image is an image that needs to be color enhanced. Exemplarily, it can be an image captured by a camera during the anchor's live broadcast, such as an image in RGB or YUV format. It can also be an image selected from an image set that needs to be color enhanced. Since color is not a single attribute, but is composed of multiple dimensions, the color enhancement process can include enhancement processes in multiple different dimensions. For example, enhancement processes for hue, saturation, brightness, contrast, etc. Among them, hue represents the basic attributes of color, such as red, green, and blue; saturation is used to measure the purity of color, high saturation means that the color is more vivid, and low saturation is expressed as grayscale; brightness affects the depth of color.

[0043] In one embodiment, image segmentation processing is performed on the input first image to obtain a plurality of segmented regions. The image segmentation processing is used to segment the image screen of the first image to obtain a plurality of segmented regions, each segmented region being a part of the first image. Optionally, the image segmentation processing may be semantic segmentation processing and / or foreground and background segmentation processing. The semantic segmentation processing is based on the semantic content of the image. For example, in the image screen containing the anchor, the face, clothes, hair, skin, seat, sofa and other regions are segmented, wherein each region is a segmented region obtained by semantic segmentation processing; for the foreground and background segmentation processing, the image is divided into a foreground region and a background region. The specific semantic segmentation processing method and the foreground and background segmentation processing method may adopt the image segmentation algorithm in the prior art, such as BiSeNet, STDC, DeepLab, MobileNet, SegNet and other semantic segmentation algorithms, RobustVideoMatting, Backgroun Matting, Deep Image Matting, Modnet and other foreground and background segmentation algorithms.

[0044] In one embodiment, a color space conversion is performed on the input first image. Optionally, the first image is subjected to color space conversion processing to obtain a second image of a preset color space mode. The preset color space mode may be a LAB color space mode and / or an HSV color space mode, or may be other color space modes, such as an HSI color space, etc., which are not strictly limited in this solution. The color space is also called a color model, which describes colors in a generally acceptable manner under certain standards. The second image is an image represented by the converted color space mode after the color space conversion of the first image. Taking the LAB color space as an example, the second image is represented by three components, L, A, and B, where L represents brightness; A represents the relative color of red and green, which is greater than 128 and tends to be red, and less than 128 and tends to be green; and B represents the relative color of yellow and blue, which is greater than 128 and tends to be yellow, and less than 128 and tends to be blue.

[0045] Step S102: Calculate and obtain a regional color evaluation value of each segmented area based on the component values ​​of the color space corresponding to the segmented area in the second image.

[0046] In one embodiment, for the multiple segmented regions obtained by the image segmentation process, the regional color evaluation value of each segmented region is calculated respectively. Optionally, the regional color evaluation value of each segmented region is calculated based on the component value of the color space of each segmented region corresponding to the second image. Taking the second image as an image in the HSV color space mode as an example, the HSV color space is a color model designed based on the perceptual attributes of color, including three components, namely the H component, the S component and the V component. The H component represents the type of color, the S component represents the saturation of the color, and the V component represents the brightness and darkness of the color. For the HSV color space, the regional color evaluation value of the segmented region is calculated based on the H component, the S component and the V component in the segmented region. The calculation method can be set accordingly according to different statistical strategies. The color evaluation value is obtained by performing color space conversion on the image and calculating the color component values ​​of the converted image. The use of this indicator can quickly, conveniently and accurately measure the color of the image for subsequent color enhancement.

[0047] Step S103: Determine a color adjustment coefficient corresponding to the regional color evaluation value through a set intensity prediction module.

[0048] In one embodiment, after obtaining the regional color evaluation value corresponding to the segmented region, a color adjustment coefficient corresponding to the regional color evaluation value is determined by a configured intensity prediction module. The color adjustment coefficient represents the degree of color enhancement during image enhancement. For example, if the color adjustment coefficient is a brightness adjustment coefficient, the brightness adjustment coefficient represents the degree of brightness enhancement of the current image.

[0049] Optionally, the intensity prediction module can be a prediction module derived from testing and analyzing the viewing time of color-enhanced images of different dimensions by users in different geographic regions. Since color adjustment is a subjective process, the implementation of color adjustment for live broadcasts is primarily determined by the color preferences of viewers in different regions, requiring experimental testing to draw conclusions. Optionally, the live broadcast audience can be divided into different countries and regions, and images with color adjustments of different dimensions, directions, and intensities can be presented to viewers in these countries and regions. Changes in viewing time can be analyzed to detect the color adjustment intensities preferred by viewers in different countries and regions, thereby generating an intensity prediction module.

[0050] Optionally, taking the regional color evaluation value as the saturation evaluation value as an example, the intensity prediction module set up can make predictions based on the saturation evaluation value to obtain a reasonable saturation adjustment coefficient that meets the user's preference, so as to be used for subsequent saturation enhancement of the image based on the saturation adjustment coefficient. In one embodiment, taking saturation as an example, when setting up the intensity prediction module, a plurality of saturation evaluation value intervals are configured, and a corresponding saturation adjustment coefficient is set for each evaluation value interval. When making a prediction, the evaluation value interval into which the saturation evaluation value falls is determined, and the saturation adjustment coefficient corresponding to the evaluation value interval is used as the adjustment coefficient for saturation enhancement of the current image. By utilizing the intensity prediction module obtained based on the experimental analysis of offline data, the color of the image can be enhanced more reasonably, so that the enhanced image is more in line with the user's preference, and more realistic and natural.

[0051] Step S104: Perform color enhancement on the first image based on the color adjustment coefficient.

[0052] In one embodiment, after determining a color adjustment coefficient, color enhancement is performed on the first image, i.e., the original input image, based on the color adjustment coefficient. Optionally, the color adjustment coefficient may be an adjustment coefficient corresponding to different dimensions, such as hue, saturation, contrast, brightness, or other dimensions, to enhance the color of the first image in the corresponding dimension. For example, if the brightness adjustment coefficient is determined to be a, then the brightness value of each pixel in the first image is multiplied by a to obtain a brightness-enhanced image. For another example, if the saturation adjustment coefficient for the host's face in the first image is b, then the original saturation of the facial area in the first image is multiplied by the saturation adjustment coefficient b to obtain a saturation-enhanced image.

[0053] As can be seen from the above, by obtaining an input first image, performing image segmentation processing on the first image to obtain multiple segmented regions, and performing color space conversion processing on the first image to obtain a second image in a preset color space mode, a regional color evaluation value for each segmented region is calculated based on the color space component values ​​of the corresponding segmented regions in the second image, a color adjustment coefficient corresponding to the regional color evaluation value is determined by a predetermined intensity prediction module, and the first image is color enhanced based on the color adjustment coefficient. In the above scheme, the image is segmented to obtain multiple segmented regions, and the color of each segmented region is evaluated using the color component values ​​after color space conversion. This can adapt to scenes with complex image content. After the color adjustment coefficient is determined by the intensity prediction module, the color adjustment coefficient is then used to achieve color enhancement of the original image. Because the color adjustment coefficient is determined by the regional color evaluation value, and different segmented regions correspond to different regional color evaluation values ​​during the determination of the regional color evaluation value, the image enhancement effect after the color enhancement is better. At the same time, the enhanced image does not directly input the original image using an end-to-end generative model, which reduces computational overhead.

[0054] FIG2 is a flow chart of another method for image color enhancement based on color evaluation provided by an embodiment of the present application, which provides an optional image segmentation processing and color space conversion method, as shown in FIG2 , including:

[0055] Step S201: obtain an input first image, perform semantic segmentation processing on the first image to obtain multiple semantic segmentation regions, and perform color space conversion processing on the first image to obtain a second image in LAB color space mode and HSV color space mode.

[0056] In one embodiment, semantic segmentation is used to perform image segmentation processing on the first image. For the original input first image, multiple semantic segmentation regions are obtained through semantic segmentation processing. For example, as shown in FIG3 , FIG3 is a schematic diagram of semantic segmentation processing of an image provided in an embodiment of the present application. After semantic segmentation of the first image, a facial region, a hair region, and a clothing region are obtained.

[0057] In one embodiment, when performing color space conversion on the first image, both the LAB color space and the HSV color space are used, wherein the LAB color space is a color model based on human eye perception, and the HSV color space is a color model designed based on color perceptual properties.

[0058] Step S202: Calculate a regional color evaluation value of each semantic segmentation region based on the LAB component value and the HSV component value corresponding to the semantic segmentation region in the second image.

[0059] In one embodiment, semantic segmentation is performed on the first image to obtain a facial region, a hair region, a skin region, a clothing region, and a seat region, and each component of the LAB color space and the HSV color space is calculated for each region to obtain a regional color evaluation value.

[0060] Alternatively, the calculation method may be to calculate the mean and standard deviation of the LAB component values ​​and HSV component values ​​for the semantically segmented regions in the second image, and determine the mean and standard deviation as the regional color evaluation value. Taking the facial region obtained through semantic segmentation as an example, the mean and standard deviation of the L component, the mean and standard deviation of the A component, the mean and standard deviation of the B component, the mean and standard deviation of the H component, the mean and standard deviation of the S component, the mean and standard deviation of the V component, and the standard deviation of the V component are calculated for the facial region. Similarly, the mean and standard deviation of each LAB component and each HSV component corresponding to the hair region, skin region, clothing region, and seat region can be calculated in sequence. The mean reflects the type, vividness, and brightness of the color in the local region, while the standard deviation represents the degree of color dispersion in the local region. By calculating the color evaluation value for each region by partitioning, it takes into account that different semantics have different priorities in human perception, and that regions with the same semantics often have the same or similar colors, thus reducing the difficulty of color evaluation.

[0061] Optionally, when calculating the regional color evaluation values ​​for different semantic segmentation areas respectively, masks of the corresponding areas can be generated according to the divided semantic segmentation areas. For example, for the hair area, the identification value corresponding to the hair part in the mask of the corresponding first image is 1, and the identification value of the non-hair part is 0. Different semantic segmentation areas are represented by their own masks, and then the regional color evaluation value corresponding to each semantic segmentation area is calculated by combining the corresponding component values ​​of the LAB and HSV color spaces.

[0062] Step S203: Determine a color adjustment coefficient corresponding to the regional color evaluation value through a set intensity prediction module.

[0063] In one embodiment, when performing corresponding predictions for regional color evaluation values ​​that are the mean and standard deviation of each component in the LAB color space and the HSV color space, the regional color evaluation values ​​corresponding to the indicators used in the intensity prediction module for prediction of different semantic segmentation regions can be selected to predict color adjustment coefficients. For example, for the facial and hair regions, the color adjustment coefficients are predicted using the mean and standard deviation of each component in the LAB color space; for the seat region, the color adjustment coefficients are predicted using the mean and standard deviation of each component in the HSV color space.

[0064] Step S204: Perform color enhancement on the first image based on the color adjustment coefficient.

[0065] From the above, it can be seen that the first image is segmented through semantic segmentation processing to obtain multiple semantic segmentation areas, and the numerical statistics of each component in the LAB color space and the HSV color space are performed for each semantic segmentation area. The statistical results are used as the regional color evaluation value, which is then used for subsequent prediction by the intensity prediction module to obtain a reasonable color adjustment coefficient. This scheme combines the human eye's saliency priority and uses semantic segmentation technology to distinguish different local image areas for color evaluation, so that the final color enhancement effect is more in line with the human eye's recognition characteristics.

[0066] FIG4 is a flow chart of another method for enhancing image color based on color evaluation provided by an embodiment of the present application, which provides a processing method for enhancing saturation, as shown in FIG4 , including:

[0067] Step S301: Acquire an input first image, perform semantic segmentation processing on the first image to obtain multiple semantic segmentation regions, and perform color space conversion processing on the first image to obtain a second image in a preset color space mode.

[0068] Step S302: Calculate a regional color evaluation value of each semantic segmentation region based on the component values ​​of the color space corresponding to the semantic segmentation region in the second image.

[0069] Step S303: Determine a color adjustment coefficient corresponding to the regional color evaluation value through a set intensity prediction module.

[0070] Step S304: When the color adjustment coefficient is a saturation adjustment coefficient, perform corresponding saturation enhancement processing on each semantic segmentation region of the first image based on the saturation adjustment coefficient corresponding to each semantic segmentation region.

[0071] In one embodiment, when the color adjustment coefficient is a saturation adjustment coefficient, the corresponding process of performing color enhancement processing on the image may be: based on the saturation adjustment coefficient corresponding to each semantic segmentation area, corresponding saturation enhancement processing is performed on each semantic segmentation area of ​​the first image. That is, for different semantic segmentation areas, saturation enhancement processing is performed using the corresponding saturation adjustment coefficient. Optionally, a fixed saturation adjustment coefficient can be determined for each semantic segmentation area to adjust the saturation of the area. The saturation adjustment coefficients corresponding to the facial area, skin area, hair area, clothing area, seat area, and sofa area are reduced in sequence. The fixed saturation adjustment coefficient corresponding to each semantic segmentation area can be derived based on the aforementioned intensity prediction module.

[0072] From the above, it can be seen that when performing color enhancement on the color parameter of saturation, different semantic segmentation areas correspond to different saturation adjustment coefficients, and saturation enhancement processing is performed on each area separately. That is, when performing color enhancement processing on an image, local areas are taken into consideration and saturation enhancement is performed separately, so that the image enhancement effect is more in line with the user's subjective needs.

[0073] FIG5 is a flow chart of another method for image color enhancement based on color evaluation provided in an embodiment of the present application, which provides a processing method for brightness enhancement, as shown in FIG5 , including:

[0074] Step S401: obtain an input first image, perform semantic segmentation and foreground-background segmentation on the first image to obtain multiple semantic segmentation areas and foreground areas and background areas, and perform color space conversion on the first image to obtain a second image in a preset color space mode.

[0075] Exemplarily, as shown in FIG6 , FIG6 is a schematic diagram obtained by the foreground-background segmentation processing provided in an embodiment of the present application, which divides the first image into a foreground area and a background area.

[0076] Step S402: Calculate a regional color evaluation value of each semantic segmentation region based on the component values ​​of the color space corresponding to the semantic segmentation region in the second image.

[0077] Step S403: Determine a color adjustment coefficient corresponding to the regional color evaluation value through a set intensity prediction module, wherein the color adjustment coefficient includes a foreground brightness adjustment coefficient and a background brightness adjustment coefficient.

[0078] In one embodiment, after obtaining the regional color evaluation value, the intensity prediction module may be used to predict the color adjustment coefficient corresponding to the regional color evaluation value. Optionally, when performing brightness enhancement, the intensity prediction module may predict the foreground brightness adjustment coefficient and the background brightness adjustment coefficient based on the brightness color evaluation value in the regional color evaluation value. That is, two adjustment coefficients may be obtained for the brightness adjustment coefficient, wherein the foreground brightness adjustment coefficient is used for brightness enhancement processing in the foreground area, and the background brightness adjustment coefficient is used for brightness enhancement processing in the background area.

[0079] Step S404 : When the color adjustment coefficient is a brightness adjustment coefficient, the brightness of the foreground area is enhanced based on the foreground brightness adjustment coefficient, and the brightness of the background area is enhanced based on the background brightness adjustment coefficient.

[0080] Optionally, based on the saliency priority of the image observed by the human eye, the foreground brightness adjustment coefficient is greater than the background brightness adjustment coefficient. In another embodiment, not only is the brightness adjustment coefficient of the foreground area and the background area simply differentiated, but the brightness adjustment coefficient corresponding to a specific semantic segmentation area, such as a facial area, can also be determined separately.

[0081] From the above, we can see that when performing brightness enhancement processing on an image, different brightness adjustment coefficients are used for the foreground area and the background area. A higher adjustment coefficient is given to the foreground area, and a lower adjustment coefficient is given to the background. This makes the enhanced image more consistent with the saliency priority when the human eye recognizes the image, and obtains an enhanced image that better meets user needs.

[0082] On the basis of the above technical solution, when the color adjustment coefficient is a contrast adjustment coefficient, the color of the first image is enhanced based on the color adjustment coefficient, including: performing contrast enhancement processing on the first image based on the contrast adjustment coefficient. Optionally, when performing contrast enhancement processing, a unified contrast enhancement processing can be performed based on the entire image. In which, when determining the contrast adjustment coefficient, the intensity prediction module can determine a global contrast adjustment coefficient based on the regional color evaluation value, and subsequently use the contrast adjustment coefficient to perform contrast enhancement processing on the image. In the above-mentioned image color enhancement method, for different enhancement dimensions, such as saturation, brightness and contrast, local and global processing methods are used for image enhancement processing respectively, and different enhancement processing methods are used for color indicators of different dimensions, so that the enhanced image effect is better.

[0083] FIG7 is a flowchart of another method for image color enhancement based on color evaluation provided in an embodiment of the present application, which provides an exemplary process of color enhancement processing for an image, as shown in FIG7 , including:

[0084] Step S501: Acquire an input first image, perform image segmentation processing on the first image to obtain a plurality of segmented regions, and perform color space conversion processing on the first image to obtain a second image in a preset color space mode.

[0085] Step S502: Calculate and obtain a regional color evaluation value of each segmented region based on the component values ​​of the color space corresponding to the segmented region in the second image.

[0086] Step S503: Determine the saturation adjustment coefficient, contrast adjustment coefficient, and brightness adjustment coefficient corresponding to the regional color evaluation value through the set intensity prediction module.

[0087] Color includes multiple dimensions such as hue, saturation, contrast, brightness, and white balance. Hue directly determines the color type, saturation indicates the vividness of the color, brightness describes the lightness and darkness of the color, contrast focuses on the difference in brightness between adjacent areas in the image, and white balance is the property of adjusting the overall color of the image to ensure that white looks real and is not affected by the color of the light source. Considering that color adjustment of live video cannot produce large color differences, this embodiment selects saturation, brightness, and contrast as the three dimensions for color adjustment to achieve image color enhancement.

[0088] Optionally, different regional color evaluation values ​​are obtained for the segmented regions of the image, and a saturation adjustment coefficient, a contrast adjustment coefficient, and a brightness adjustment coefficient are predicted based on the regional color evaluation values ​​by a set intensity prediction module. Optionally, the saturation adjustment coefficient is a local adjustment mode, that is, different saturation adjustment coefficients are obtained for different local regions; the contrast adjustment coefficient is a unified contrast adjustment for the entire image; and the brightness adjustment coefficient can have different adjustment coefficients for the facial area, foreground area, and background area in the image.

[0089] Step S504: generating a saturation coefficient mask based on the saturation adjustment coefficient, generating a contrast coefficient mask based on the contrast adjustment coefficient, and generating a brightness coefficient mask based on the brightness adjustment coefficient.

[0090] In one embodiment, after obtaining the saturation adjustment coefficient, contrast adjustment coefficient, and brightness adjustment coefficient, the corresponding masks are generated accordingly, namely, the saturation coefficient mask, the contrast coefficient mask, and the brightness coefficient mask. Each coefficient mask records the adjustment coefficient of the corresponding dimension. For example, as shown in Figure 8, Figure 8 is a schematic diagram of a saturation coefficient mask provided in an embodiment of the present application, wherein segmented area 1 and segmented area 2 are different segmented areas obtained based on semantic segmentation processing. Assuming that the saturation adjustment coefficient of segmented area 1 is determined to be 1.1 and the saturation adjustment coefficient of segmented area 2 is determined to be 1.2, the corresponding saturation coefficient mask schematic diagram is shown in Figure 8.

[0091] Step S505: Input the first image and the saturation coefficient mask into a saturation renderer for saturation enhancement to obtain a first enhanced image; input the first enhanced image and the contrast coefficient mask into a contrast renderer for contrast enhancement to obtain a second enhanced image; input the second enhanced image and the brightness coefficient mask into a brightness renderer for brightness enhancement to obtain a color enhanced image.

[0092] In one embodiment, saturation enhancement, contrast enhancement, and brightness enhancement of an image are respectively achieved by setting a saturation renderer, a contrast renderer, and a brightness renderer, wherein the saturation renderer, the contrast renderer, and the brightness renderer are connected in series. For example, as shown in FIG9 , FIG9 is a schematic diagram of a framework of an image color enhancement method based on color evaluation provided by an embodiment of the present application, wherein image segmentation processing is performed on the input first image to obtain a semantic segmentation map, and color evaluation and intensity prediction are performed on the first image at the same time. The obtained color adjustment coefficient is combined with the semantic segmentation map to generate a saturation coefficient mask, and the saturation coefficient mask and the first image are input to the saturation renderer to enhance the saturation of the image to obtain a first enhanced image, and then the first enhanced image and the contrast coefficient mask are input to the contrast renderer to enhance the contrast to obtain a second enhanced image, and then the second enhanced image and the brightness coefficient mask are input to the brightness renderer to enhance the brightness to obtain a color enhanced image. The contrast coefficient mask is obtained by combining the contrast adjustment coefficient obtained by intensity prediction with the entire image area, and the brightness coefficient mask is obtained by combining the brightness adjustment coefficient obtained by intensity prediction with the foreground and background segmentation map.

[0093] In one embodiment, the above-mentioned renderer is constructed using a neural network model. Optionally, the renderer as a whole adopts an encoder-decoder architecture. Taking the first image as an RGB image as an example, the encoder part of the renderer converts the input first image, that is, the three-dimensional RGB image, into a high-dimensional feature vector through a fully connected layer, and then extracts detail features through three layers of convolution. Finally, the feature vector is converted back to the three-dimensional RGB space through a decoder containing two fully connected layers. The saturation renderer, contrast renderer, and brightness renderer are trained separately and finally connected in series. The saturation, contrast, and brightness of the first image are enhanced in sequence to obtain a color-enhanced image.

[0094] From the above, we can see that the neural network renderer is set up to enhance the saturation, contrast and brightness of the image in turn, realizing color adjustment of saturation, contrast and brightness. This color enhancement solution is applied to the live broadcast business, which significantly increases the average viewing time per person and achieves positive benefits.

[0095] Figure 10 is a block diagram of a device for color enhancement based on color evaluation provided in an embodiment of the present application. The device is configured to execute the method for color enhancement based on color evaluation provided in the above embodiment and has the corresponding functional modules and beneficial effects. As shown in Figure 10, the device includes:

[0096] An image acquisition module 101 is configured to acquire an input first image;

[0097] An image segmentation module 102 is configured to perform image segmentation processing on the first image to obtain a plurality of segmented regions;

[0098] An image conversion module 103 is configured to perform color space conversion processing on the first image to obtain a second image in a preset color space mode;

[0099] An evaluation value calculation module 104 is configured to calculate a regional color evaluation value of each segmented area based on a component value of a color space corresponding to the segmented area in the second image;

[0100] an intensity prediction module 105 configured to determine a color adjustment coefficient corresponding to the color evaluation value of the region;

[0101] The color adjustment module 106 is configured to perform color enhancement on the first image based on the color adjustment coefficient.

[0102] In the above scheme, a first image is obtained as an input, image segmentation is performed on the first image to obtain multiple segmented regions, and color space conversion is performed on the first image to obtain a second image in a preset color space mode. Based on the color space component values ​​of the corresponding segmented regions in the second image, a regional color evaluation value is calculated for each segmented region. A color adjustment coefficient corresponding to the regional color evaluation value is determined by a predetermined intensity prediction module, and the first image is color enhanced based on the color adjustment coefficient. In the above scheme, the image is segmented to obtain multiple segmented regions, and the color of each segmented region is evaluated using the color component values ​​after color space conversion. This can adapt to scenes with complex image content. After the color adjustment coefficient is determined by the intensity prediction module, the color adjustment coefficient is then used to achieve color enhancement of the original image. Because the color adjustment coefficient is determined by the regional color evaluation value, and different segmented regions correspond to different regional color evaluation values ​​during the determination of the regional color evaluation value, the final image enhancement effect after color enhancement is better. At the same time, the enhanced image is not directly input into the original image using an end-to-end generative model, which reduces computational overhead.

[0103] In a possible embodiment, the preset color space mode includes a LAB color space mode and / or an HSV color space mode, and the image segmentation module 102 is configured to:

[0104] Performing semantic segmentation processing on the first image to obtain a plurality of semantic segmentation regions;

[0105] The evaluation value calculation module 104 is configured as follows:

[0106] A regional color evaluation value of each semantic segmentation region is calculated based on the LAB component value and / or HSV component value corresponding to the semantic segmentation region in the second image.

[0107] In a possible embodiment, the evaluation value calculation module 104 is configured as follows:

[0108] Calculating the mean and standard deviation of the LAB component values ​​and / or HSV component values ​​corresponding to the semantic segmentation area in the second image respectively;

[0109] The mean value and the standard deviation are determined as regional color evaluation values.

[0110] In a possible embodiment, the intensity prediction module is obtained based on testing and analysis of viewing time of color-enhanced images of different dimensions by users in different geographical areas, and the color adjustment coefficient includes any one or more of a saturation adjustment coefficient, a contrast adjustment coefficient, and a brightness adjustment coefficient.

[0111] In a possible embodiment, when the color adjustment coefficient is a saturation adjustment coefficient, different semantic segmentation regions correspond to different saturation adjustment coefficients, and the color adjustment module 106 is configured as follows:

[0112] Based on the saturation adjustment coefficient corresponding to each semantic segmentation area, corresponding saturation enhancement processing is performed on each semantic segmentation area of ​​the first image.

[0113] In a possible embodiment, when the color adjustment coefficient is a contrast adjustment coefficient, the color adjustment module 106 is configured as follows:

[0114] Performing contrast enhancement processing on the first image based on the contrast adjustment coefficient.

[0115] In a possible embodiment, when the color adjustment coefficient is a brightness adjustment coefficient, the image segmentation module 103 is configured as follows:

[0116] Performing foreground-background segmentation processing on the first image to obtain a foreground area and a background area, wherein the brightness adjustment coefficient includes a foreground brightness adjustment coefficient and a background brightness adjustment coefficient;

[0117] The color adjustment module 106 is configured as follows:

[0118] The brightness of the foreground area is adjusted for enhancement based on the foreground brightness adjustment coefficient, and the brightness of the background area is adjusted for enhancement based on the background brightness adjustment coefficient.

[0119] In a possible embodiment, when the color adjustment coefficient is a saturation adjustment coefficient, a contrast adjustment coefficient, and a brightness adjustment coefficient, the color adjustment module 106 is configured as follows:

[0120] generating a saturation coefficient mask based on the saturation adjustment coefficient, generating a contrast coefficient mask based on the contrast adjustment coefficient, and generating a brightness coefficient mask based on the brightness adjustment coefficient;

[0121] Inputting the first image and the saturation coefficient mask into a saturation renderer to perform saturation enhancement to obtain a first enhanced image;

[0122] Inputting the first enhanced image and the contrast coefficient mask into a contrast renderer for contrast enhancement to obtain a second enhanced image;

[0123] The second enhanced image and the brightness coefficient mask are input into a brightness renderer for brightness enhancement to obtain a color enhanced image.

[0124] FIG11 is a schematic diagram of the structure of a color evaluation-based image enhancement device provided in an embodiment of the present application. As shown in FIG11 , the device includes a processor 201, a memory 202, an input device 203, and an output device 204. The device may include one or more processors 201, with FIG11 assuming a single processor 201 as an example. The processor 201, memory 202, input device 203, and output device 204 may be connected via a bus or other means, with FIG11 assuming a bus connection as an example. Memory 202, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the color evaluation-based image enhancement method in the embodiment of the present application. Processor 201 executes the software programs, instructions, and modules stored in memory 202 to execute various functional applications and data processing of the device, thereby implementing the aforementioned color evaluation-based image enhancement method. Input device 203 can be configured to receive input digital or character information and generate key signal input related to user settings and function control of the device. Output device 204 may include a display device such as a display screen.

[0125] An embodiment of the present application further provides a non-volatile storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are configured to perform an image color enhancement method based on color evaluation as described in the above embodiment, which includes:

[0126] Acquire an input first image, perform image segmentation processing on the first image to obtain a plurality of segmented regions, and perform color space conversion processing on the first image to obtain a second image in a preset color space mode;

[0127] Calculating a regional color evaluation value of each of the segmented regions based on component values ​​of the color space corresponding to the segmented regions in the second image;

[0128] Determine the color adjustment coefficient corresponding to the color evaluation value of the region by using a set intensity prediction module;

[0129] Color enhancement is performed on the first image based on the color adjustment coefficient.

[0130] It is worth noting that in the above-mentioned embodiment of the image color enhancement device based on color evaluation, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the various functional units are only for the convenience of distinguishing each other, and are not configured to limit the scope of protection of the embodiments of the present application.

[0131] In some possible implementations, various aspects of the methods provided herein may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is configured to cause the computer device to perform the steps of the methods according to the various exemplary embodiments of the present application described above. For example, the computer device may perform the image color enhancement method based on color assessment described in the embodiments of the present application. The program product may be implemented using any combination of one or more readable media.

Claims

1. An image color enhancement method based on color evaluation, wherein, Including: Obtaining an input first image, performing image segmentation processing on the first image to obtain a plurality of segmentation regions, and performing color space conversion processing on the first image to obtain a second image in a preset color space mode; Calculating a regional color evaluation value for each of the segmentation regions based on the component values of the color space corresponding to the segmentation regions in the second image; Determining a color adjustment coefficient corresponding to the regional color evaluation value through a set intensity prediction module; Performing color enhancement on the first image based on the color adjustment coefficient.

2. The method for image color enhancement based on color evaluation according to claim 1, wherein, The preset color space mode includes the LAB color space mode and / or the HSV color space mode. The performing image segmentation processing on the first image to obtain a plurality of segmentation regions includes: Performing semantic segmentation processing on the first image to obtain a plurality of semantic segmentation regions; Correspondingly, the calculating a regional color evaluation value for each of the segmentation regions based on the component values of the color space corresponding to the segmentation regions in the second image includes: Calculating a regional color evaluation value for each of the semantic segmentation regions based on the LAB component values and / or the HSV component values corresponding to the semantic segmentation regions in the second image.

3. The method for enhancing image color based on color evaluation according to claim 2, wherein, The calculating a regional color evaluation value for each of the semantic segmentation regions based on the LAB component values and / or the HSV component values corresponding to the semantic segmentation regions in the second image includes: Calculating the mean and standard deviation of the LAB component values and / or the HSV component values corresponding to the semantic segmentation regions in the second image respectively; Determining the mean and the standard deviation as the regional color evaluation value.

4. The method for enhancing image color based on color evaluation according to any one of claims 1-3, wherein, The intensity prediction module is obtained through test analysis of the viewing duration of users in different geographical regions for color-enhanced images in different dimensions. The color adjustment coefficient includes any one or more of a saturation adjustment coefficient, a contrast adjustment coefficient, and a brightness adjustment coefficient.

5. The method for enhancing image color based on color evaluation according to claim 4, wherein, In the case where the color adjustment coefficient is a saturation adjustment coefficient, different semantic segmentation regions correspond to different saturation adjustment coefficients. The performing color enhancement on the first image based on the color adjustment coefficient includes: Performing corresponding saturation enhancement processing on each semantic segmentation region of the first image based on the saturation adjustment coefficient corresponding to each semantic segmentation region.

6. The method for image color enhancement based on color evaluation according to claim 4, wherein, In the case where the color adjustment coefficient is a contrast adjustment coefficient, the performing color enhancement on the first image based on the color adjustment coefficient includes: Performing contrast enhancement processing on the first image based on the contrast adjustment coefficient.

7. The method for enhancing image color based on color evaluation according to claim 4, wherein, In the case where the color adjustment coefficient is a brightness adjustment coefficient, the performing image segmentation processing on the first image to obtain a plurality of segmentation regions includes: Performing foreground and background segmentation processing on the first image to obtain a foreground region and a background region. The brightness adjustment coefficient includes a foreground brightness adjustment coefficient and a background brightness adjustment coefficient; Correspondingly, the performing color enhancement on the first image based on the color adjustment coefficient includes: Performing brightness enhancement adjustment on the foreground region based on the foreground brightness adjustment coefficient, and performing brightness enhancement adjustment on the background region based on the background brightness adjustment coefficient.

8. The method for enhancing image color based on color evaluation according to any one of claims 1-4, wherein, When the color adjustment coefficients are the saturation adjustment coefficient, the contrast adjustment coefficient, and the brightness adjustment coefficient, the color enhancement of the first image based on the color adjustment coefficients includes: generating a saturation coefficient mask based on the saturation adjustment coefficient, generating a contrast coefficient mask based on the contrast adjustment coefficient, and generating a brightness coefficient mask based on the brightness adjustment coefficient; inputting the first image and the saturation coefficient mask into a saturation renderer for saturation enhancement to obtain a first enhanced image; inputting the first enhanced image and the contrast coefficient mask into a contrast renderer for contrast enhancement to obtain a second enhanced image; inputting the second enhanced image and the brightness coefficient mask into a brightness renderer for brightness enhancement to obtain a color-enhanced image.

9. An image color enhancement device based on color evaluation, wherein, including: an image acquisition module configured to acquire an input first image; an image segmentation module configured to perform image segmentation processing on the first image to obtain a plurality of segmentation regions; an image conversion module configured to perform color space conversion processing on the first image to obtain a second image in a preset color space mode; an evaluation value calculation module configured to calculate a regional color evaluation value for each of the segmentation regions based on the component values of the color space corresponding to the segmentation regions in the second image; an intensity prediction module configured to determine a color adjustment coefficient corresponding to the regional color evaluation value; a color adjustment module configured to perform color enhancement on the first image based on the color adjustment coefficient.

10. An image color enhancement device based on color evaluation, the device comprising: one or more processors; a storage device configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the color evaluation-based image color enhancement method according to any one of claims 1-8.

11. A non-volatile storage medium storing computer-executable instructions, the computer-executable instructions being configured to execute the color evaluation-based image color enhancement method according to any one of claims 1-8 when executed by a computer processor.

12. A computer program product, comprising a computer program, wherein, The computer program, when executed by a processor, implements the color evaluation-based image color enhancement method according to any one of claims 1-8.

Images