Image color enhancement method, device, computer equipment, medium and product

Abstract

The embodiments of the present application disclose a method, device, computer equipment, medium and product for color enhancement of an image, and relate to the field of image processing. The method comprises: obtaining the pixel color value of a pixel point in an image in a color space, the color space comprising a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area; determining the pixel color enhancement coefficient of the pixel point according to the color enhancement mapping relationship and the pixel color value, the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficients corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area; enhancing the color of each pixel point in the image according to the pixel color enhancement coefficient to obtain a color enhanced image. In the embodiments of the present application. By setting a transition area, the present application avoids problems such as color omission and color jump in color enhancement, and improves the quality of the color enhanced image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to a method, apparatus, computer equipment, medium and product for enhancing the color of an image. Background Art

[0002] With the development of color enhancement technology, computer devices can perform color enhancement operations on pixels in image frames (such as color enhancement operations that increase contrast or saturation), thereby improving the visual effects of image frames. However, if the same intensity of color enhancement operations are performed on each pixel in the image frame, the display effect of some pixels may deteriorate. For example, pixels in skin color areas such as faces may appear yellowish, reddish, or blackish.

[0003] In the related technology, the pixel value of the pixel point can be mapped to the color space, and the color space can be divided into a skin color area and a non-skin color area according to a color threshold; the pixel points corresponding to the skin color area are color enhanced, and the pixel points corresponding to the non-skin color area are not processed or are subjected to a weaker degree of color enhancement.

[0004] However, the above method has some disadvantages. For example, the determination of the color threshold may be biased, resulting in pixels that should belong to the skin color area being classified as non-skin color areas, resulting in missed detection; for another example, simple division based on the color threshold may result in discontinuity and jumps in the color of the pixels after color enhancement. Summary of the invention

[0005] The embodiments of the present application provide a method, device, computer equipment, medium and product for enhancing the color of an image. The technical solution is as follows:

[0006] On the one hand, an embodiment of the present application provides a method for color enhancement of an image, the method comprising:

[0007] Obtaining a pixel color value of each pixel in the image in a color space, wherein the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area;

[0008] Determine a pixel color enhancement coefficient for each pixel point according to a color enhancement mapping relationship and the pixel color value, wherein the color enhancement mapping relationship represents a correspondence between a color value in the color space and a color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area;

[0009] According to the pixel color enhancement coefficient of each pixel point, the color of each pixel point in the image is enhanced to obtain a color enhanced image.

[0010] On the other hand, an embodiment of the present application provides a device for enhancing color of an image, the device comprising:

[0011] An acquisition module, used to acquire a pixel color value of each pixel in the image in a color space, wherein the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area;

[0012] A determination module, configured to determine a pixel color enhancement coefficient of each pixel point according to a color enhancement mapping relationship and the pixel color value, wherein the color enhancement mapping relationship represents a correspondence between a color value in the color space and a color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area;

[0013] The color enhancement module is used to enhance the color of each pixel in the image according to the pixel color enhancement coefficient of each pixel to obtain a color enhanced image.

[0014] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein at least one computer instruction is stored in the memory, and the at least one computer instruction is loaded and executed by the processor to implement the method described in any of the above embodiments.

[0015] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer instruction is stored. The computer instruction is loaded and executed by a processor to implement the method described in the above aspects.

[0016] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above aspects.

[0017] In the embodiment of the present application, the color space includes a color protection area, a non-color protection area and a transition area. Since the transition area is located between the color protection area and the non-color protection area, the color value to be protected has a high probability of falling into the color protection area, a low probability of falling into the transition area, and a very low probability of falling into the non-color protection area, thereby avoiding color omissions. At the same time, since the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area, after color enhancement, the color enhancement effect of the color protection area can be gradually converted into the color enhancement effect of the non-color protection area in a transitional manner, rather than abruptly split and changed, thereby avoiding color discontinuity and jumps, thereby improving the quality of the color enhanced image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is a computer system architecture diagram provided by an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for color enhancement of an image provided by an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a UV plane in a YUV color space provided by an exemplary embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a color protection area, a transition area, and a non-color protection area in a UV plane provided by an exemplary embodiment of the present application;

[0023] Figure 5 is a schematic diagram of determining a first rectangular area and a second rectangular area provided by an exemplary embodiment of the present application;

[0024] Figure 6 is a schematic diagram of a modified transition region provided by an exemplary embodiment of the present application;

[0025] Figure 7 is a schematic diagram of the distribution of color enhancement coefficients provided by an exemplary embodiment of the present application;

[0026] Figure 8 is a schematic diagram of a first ellipse provided by an exemplary embodiment of the present application;

[0027] Fig. 9 is a schematic diagram of a second ellipse provided by an exemplary embodiment of the present application;

[0028] Fig.10 is a schematic diagram of the regional distribution of transition areas before and after saturation enhancement provided by an exemplary embodiment of the present application;

[0029] Fig.11 It is a flowchart of determining a third color enhancement coefficient corresponding to a color value in a transition area according to a first color enhancement coefficient, a second color enhancement coefficient, and a distance from a color value in a transition area to a color protection area provided by an exemplary embodiment of the present application;

[0030] Fig.12 is a schematic diagram of an affine transformation relationship provided by an exemplary embodiment of the present application;

[0031] Fig.13 is a schematic diagram of the distribution of a third color enhancement coefficient provided by an exemplary embodiment of the present application;

[0032] Fig.14 It is a schematic diagram of the distribution of color enhancement coefficients when protecting skin color pixels and high saturation pixels at the same time, provided by an exemplary embodiment of the present application;

[0033] Fig.15 is a schematic diagram of a terminal display color enhancement control provided by an exemplary embodiment of the present application;

[0034] Fig.16 is a flowchart of the construction phase and application phase of the color enhancement mapping relationship provided by an exemplary embodiment of the present application;

[0035] Fig.17 is a structural block diagram of an image color enhancement device provided by an exemplary embodiment of the present application;

[0036] Fig.18 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0038] First, the terms involved in the embodiments of the present application are introduced.

[0039] Video enhancement: A technology that uses digital signal processing methods to process and optimize video images. Its main purpose is to improve the quality of the image, making the video clearer, more vivid, and smoother. Video enhancement includes multiple dimensions such as clarity enhancement, color enhancement, and frame rate enhancement. This application focuses on color enhancement, that is, improving the color effect of the image. The method provided in this application can be used to perform color enhancement on one or more frames in a video, and can also be used to perform color enhancement on other types of images in non-video scenes.

[0040] Color enhancement: A technique or processing method that adjusts and optimizes the color of an image to improve visual effects, highlight specific information, or meet specific needs. Color enhancement can make the color of an image more vivid, rich, and lively. Color enhancement includes, but is not limited to, contrast enhancement, saturation enhancement, brightness adjustment, white balance adjustment, and one or more other color enhancement methods.

[0041] Contrast: The degree of difference between light and dark areas in an image. The higher the contrast, the greater the difference between light and dark areas in an image; the lower the contrast, the smaller the difference between light and dark areas in an image.

[0042] Saturation: The vividness of a color, also known as the purity of a color. Saturation depends on the ratio of the colored component to the achromatic component (i.e., gray). The larger the colored component, the greater the saturation, and the brighter the color of the image or video screen; the larger the achromatic component, the smaller the saturation, and the duller the color of the image or video screen.

[0043] Color space: represents the various colors that human vision can perceive through points in a one-dimensional or multi-dimensional (including two-dimensional, three-dimensional or four-dimensional) coordinate system. Common color spaces include RGB color space, HSV color space, YCbCr color space, YUV color space, Lab color space, etc.

[0044] RGB color space: It is established in a three-dimensional Cartesian coordinate system (i.e. a three-dimensional rectangular coordinate system) and represents colors as cubes of unit length. The three coordinate axes are based on the three basic colors of red, green, and blue, and colors are represented as the superposition of the three colors of red, green, and blue.

[0045] YUV color space: represents color as brightness parameter (Y: Luminance or Luma) and chrominance parameter (U, V: Chrominance or Chroma). U and V are used to describe the color and saturation of the image and are used to specify the color of the pixel.

[0046] Region Of Interest (ROI): In fields such as machine vision and image processing, the area to be processed is outlined in the image in the form of a box, circle, ellipse, irregular polygon, etc., which is called the ROI. For example, the skin color area in the image is the ROI, or the high saturation area in the image is the ROI.

[0047] Through a lot of practice and analysis, human skin color is generally distributed in a specific area of ​​the color space. Therefore, a variety of skin color detection methods based on color space thresholds have been developed, that is, the image pixel values ​​are mapped to different color spaces. If the color threshold is met, it is determined to be skin color, otherwise it is determined to be non-skin color. For example, in the related art, a skin color detection method based on the HSV color space and a skin color detection method based on the RGB color space can be used to determine the skin color area in the image. In the related art, based on the results of skin color detection, color enhancement is performed on non-skin color pixels, and the skin color area is not processed or a weaker degree of color enhancement is performed, which can achieve color protection of the skin color area and avoid problems such as reddish, yellowish or blackish skin color areas after color enhancement.

[0048] However, the effectiveness of the above methods is highly dependent on the accuracy and stability of the skin color detection method. If the color threshold is not selected properly, it may lead to skin color missed detection and color jump. At the same time, for videos, the skin color detection results of adjacent frames need to be as consistent as possible, otherwise it is very easy for the timing of adjacent frames after color enhancement to be unstable, resulting in poor color enhancement effect.

[0049] In order to avoid problems such as color omission and color jump in color enhancement and improve the quality of color enhanced images, the present application provides a color enhancement method for an image.

[0050] Application scenarios of image color enhancement methods include but are not limited to the following.

[0051] (1) Color enhancement scenarios for video applications.

[0052] In a possible scenario, the color enhancement method of the image may be executed by a server.

[0053] Exemplarily, the server obtains video frames from the video, and uses the image color enhancement method provided in the present application to perform color enhancement processing on the video frames to obtain a color-enhanced video.

[0054] Exemplarily, the terminal plays the video before color enhancement through the video application, and when the user instructs to perform a color enhancement operation, the terminal obtains the color-enhanced video from the server and displays it. For example, when the user clicks the color enhancement control displayed on the terminal, or triggers the shortcut key corresponding to color enhancement, the terminal obtains the color-enhanced video from the server and displays it.

[0055] (2) Color enhancement scenarios in photo-enhancing or social applications.

[0056] In a possible scenario, the method for enhancing the color of an image may be executed by a terminal.

[0057] Exemplarily, the terminal obtains real-time or historical photos from an album or camera, and uses the image color enhancement method provided in the present application to perform color enhancement processing on the photos to obtain color-enhanced photos to improve the color display effect of the photos, and stores the color-enhanced photos in the album.

[0058] Exemplarily, the terminal obtains photos from a beauty application or a social application, and uses the image color enhancement method provided in the present application to perform color enhancement processing on the photos to obtain color-enhanced photos, and replaces the photos in the beauty application or the social application with the color-enhanced photos, thereby improving the color display effect of the photos.

[0059] (3) Color enhancement scenarios for live broadcast applications.

[0060] In a possible scenario, the color enhancement method of the image may be executed by a server.

[0061] Exemplarily, the server obtains video frames from real-time streaming media, and uses the image color enhancement method provided in the present application to perform color enhancement processing on the video frames to obtain real-time streaming media after color enhancement.

[0062] Exemplarily, the terminal plays the real-time streaming media before color enhancement through a live broadcast application. When the user performs a color enhancement operation through an operation instruction, the terminal obtains the real-time streaming media after color enhancement from the server in real time and displays it in real time.

[0063] It should be noted that the above scenarios are only examples, and those skilled in the art may apply the image color enhancement method proposed in this application to any other possible scenarios as needed, without limitation.

[0064] See also Figure 1 , Figure 1 1 is a computer system architecture diagram provided by an exemplary embodiment of the present application. The computer system includes a terminal 110 and a server 120.

[0065] The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a smart watch, a car terminal, or the like, which has image display capabilities or video playback capabilities, but is not limited thereto.

[0066] The terminal 110 and the server 120 may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.

[0067] The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In addition, the server 120 may also be a node server in a blockchain network.

[0068] In a possible scenario, the terminal 110 may be used to execute the image color enhancement method proposed in the present application. For example, the terminal 110 may first display the image before color enhancement, and when receiving a color enhancement operation indicated by the user, the terminal 110 uses the image color enhancement method proposed in the present application to enhance the color of each pixel in the image, and displays the color enhanced image.

[0069] In another possible scenario, the server 120 may be used to execute the image color enhancement method proposed in the present application. For example, the terminal 110 may first display the image before color enhancement, and upon receiving the color enhancement operation indicated by the user, the terminal 110 sends a color enhancement request to the server 120. The server 120 uses the image color enhancement method proposed in the present application to enhance the color of each pixel in the image according to the color enhancement request, obtains a color enhanced image, and transmits the color enhanced image to the terminal 110, which then displays the color enhanced image.

[0070] In other possible scenarios, the terminal 110 and the server 120 may jointly execute the image color enhancement method provided by the present application. For example, the steps of obtaining the pixel color value and determining the pixel color enhancement coefficient may be executed by the server 120, and the step of performing color enhancement processing according to the pixel color enhancement coefficient may be executed by the terminal 110, and the embodiments of the present application are not limited to this.

[0071] It should be noted that the above embodiment only describes the general architecture of the computer system. The system may also include more or fewer components, or combine certain components, and this embodiment does not limit this.

[0072] See also Figure 2 , Figure 2 FIG. 1 is a flowchart of a method for color enhancement of an image provided by an exemplary embodiment of the present application. In some embodiments, the method is performed by a computer device. Optionally, the computer device may be Figure 1 The terminal or server in the computer system shown in the figure. The method comprises the following steps.

[0073] Step 201, obtaining the pixel color value of each pixel in the image in the color space, where the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area.

[0074] Optionally, in a color enhancement scenario of a video application, the image may be a video frame obtained from a video.

[0075] Optionally, in a color enhancement scenario of a beauty application or a social application, the image may be a real-time or historical photo obtained from a photo album or camera of the terminal, or the image may be a photo obtained from a beauty application or a social application.

[0076] Optionally, in a color enhancement scenario of a live broadcast application, the image may be a video frame obtained from a real-time streaming media.

[0077] In addition, the images may also be obtained from any other possible means without any limitation.

[0078] It should be noted that in the process of collecting relevant data (such as images) of the user, the present application may display a prompt interface, a pop-up window or output a voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that the relevant data is currently being collected, so that the present application only starts to execute the relevant steps of obtaining the relevant data of the user after obtaining the confirmation operation issued by the user on the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining the relevant data of the user are terminated, that is, the relevant data of the user is not obtained. In other words, the information (including but not limited to user device information, user personal information, user's real-time location), data (including but not limited to data for analysis, data for storage, data for display, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the data such as images involved in the present application are all obtained with full authorization.

[0079] In some embodiments, the color space is a YUV color space, and the pixel color value of a pixel in the image in the color space includes at least one of a color value Y, a color value U, and a color value V. Optionally, the value range of the color value Y, the color value U, and the color value V is an integer between 0 and 255.

[0080] In some embodiments, the color space may also be other types of color spaces, such as RGB color space, HSV color space, YCbCr color space, Lab color space, etc., without limitation thereto.

[0081] When the color space is a YUV color space, regarding the specific method of obtaining the pixel color value of a pixel point in the image in the color space, the computer device can determine the pixel color value of the pixel point in the color space based on the RGB value of the pixel point and the conversion formula between the RGB value and the YUV value (Formula 1).

[0082] ;(Formula 1)

[0083] For example, when the RGB value of a pixel is (0, 0, 0), the pixel color value of the pixel in the YUV color space includes a color value Y of 0, a color value U of 127, and a color value V of 127.

[0084] In some embodiments, the computer device obtains the pixel color value (including the color value U and the color value V) of the pixel point in the image in the UV plane in the YUV color space.

[0085] The UV plane is a plane with the color value U as the horizontal coordinate and the color value V as the vertical coordinate.

[0086] See also Figure 3 , Figure 3 It is a schematic diagram of a UV plane in a YUV color space provided by an exemplary embodiment of the present application.

[0087] like Figure 3 As shown, the UV plane in the YUV color space is a UV plane 310 composed of the color value U as the horizontal coordinate (the value range of the color value U is an integer from 0 to 255) and the color value V as the vertical coordinate (the value range of the color value V is an integer from 0 to 255).

[0088] In the UV plane 310, not all regions have colors. The region with colors in the UV plane 310 is the chromaticity region 320. Figure 3 As shown, the chromaticity region 320 is displayed as a hexagonal region. The chromaticity region 320 includes the color value U and the color value V corresponding to each color.

[0089] Among them, the center point (128, 128) of the UV plane is the point with the lowest color saturation in the chromaticity area 220 (corresponding to gray). With the center point (128, 128) as the vertex, rays are drawn in all directions. The farther the point on the same ray is from the vertex (128, 128), the higher the color saturation.

[0090] In some embodiments, the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area.

[0091] Optionally, the color protection area, transition area, and non-color protection area in the color space may be a simply connected area (such as a circle, an ellipse, or a rectangle), or a multiply connected area (such as an annular area between two concentric circles of different radii, an elliptical annular area between two concentric ellipses, or a strip area between two rectangles). Regarding the specific shapes of the color protection area, the non-color protection area, and the transition area, those skilled in the art may also determine any other possible shapes according to actual needs, and no limitation is made thereto.

[0092] In some embodiments, the transition region is between the color protection region and the non-color protection region, that is, the distance from the transition region to the color protection region is smaller than the distance from the non-color protection region to the color protection region. The color protection region is a region consisting of color values ​​to be protected in the color space, and the non-color protection region is a region consisting of non-protected color values ​​in the color space. In the color enhancement process, the color enhancement degree of the color values ​​to be protected is different from the color enhancement degree of the non-protected color values; and the color enhancement degree of the color values ​​in the transition region is between the color enhancement degree of the color values ​​to be protected in the color protection region and the color enhancement degree of the non-protected color values ​​in the non-color protection region.

[0093] For example only, in the color enhancement scenario for skin color protection, the color protection area is the skin color area, the transition area is the area close to the skin color, and the non-color protection area is the non-skin color area. In the color enhancement process, in order to protect the skin color from deviation, the color enhancement degree of the color value to be protected in the color protection area is less than the color enhancement degree of the color value in the transition area; and the color enhancement degree of the color value in the transition area is less than the color enhancement degree of the non-protected color value in the non-color protection area.

[0094] For pixels at the skin position in the image, the pixel color value in the color space has a high probability of falling into the skin color area (color protection area), a small probability of falling into the near skin color area (transition area), and a very low probability of falling into the non-skin color area (non-color protection area). Therefore, the near skin color area (transition area) can ensure that skin pixels will not be missed, avoiding unnatural problems such as reddish or yellowish skin after color enhancement.

[0095] For example only, in the color enhancement scenario for high saturation protection, the color protection area is a high saturation area, the transition area is a near high saturation area, and the non-color protection area is a low saturation area. In the color enhancement (saturation enhancement) process, in order to ensure that the high saturation color is not over-enhanced, the saturation enhancement degree of the color value to be protected in the color protection area is less than the saturation enhancement degree of the color value in the transition area; and the saturation enhancement degree of the color value in the transition area is less than the saturation enhancement degree of the non-protected color value in the non-color protection area.

[0096] For example only, in a color enhancement scenario optimized for food display effects, the color protection area is a food color area, the transition area is a near-food color area, and the non-color protection area is a non-food color area. During the color enhancement process, in order to optimize the display effect of food, the degree of saturation enhancement of the color value to be protected in the color protection area is greater than the degree of saturation enhancement of the color value in the transition area; and the degree of saturation enhancement of the color value in the transition area is greater than the degree of saturation enhancement of the non-protected color value in the non-color protection area.

[0097] See also Figure 4 , Figure 4 It is a schematic diagram of a color protection area, a transition area, and a non-color protection area in a UV plane provided by an exemplary embodiment of the present application.

[0098] like Figure 4 As shown, in the UV plane 410, the chromaticity region 420 having color is displayed as a hexagonal region. In the chromaticity region 420, the color protection region is displayed as a smaller irregular region 430, the transition region is displayed as a region located within the larger irregular region 440 and outside the smaller irregular region 430, and the non-color protection region is displayed as a region located within the chromaticity region 420 and outside the larger irregular region 440. That is, the transition region is located between the color protection region and the non-color protection region.

[0099] Step 202, according to the color enhancement mapping relationship and the pixel color value, the pixel color enhancement coefficient of each pixel point is determined, the color enhancement mapping relationship represents the correspondence between the color value in the color space and the color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area.

[0100] Optionally, the color enhancement coefficient is used to indicate the degree of color enhancement after color enhancement. For example, the larger the color enhancement coefficient, the greater the degree of color enhancement after color enhancement. For example, in a saturation enhancement scenario, a color enhancement coefficient of 1 represents no saturation enhancement, a color enhancement coefficient of 1.2 represents a smaller degree of saturation enhancement, and a color enhancement coefficient of 1.35 represents a larger degree of saturation enhancement.

[0101] Optionally, the color enhancement mapping relationship is preconfigured and determined by the computer device. In some embodiments, the computer device divides the color space into a color protection area, a transition area, and a non-color protection area, and assigns a corresponding color enhancement coefficient to each area.

[0102] In a possible implementation, the color enhancement mapping relationship is stored through a mapping table; the mapping table contains the corresponding relationship between each color value (U, V) in the color space and the color enhancement coefficient, such as the color enhancement coefficient corresponding to the center point (128, 128) is 1.5, and the color enhancement coefficient corresponding to the point (100, 150) with a color value U and a color value V of 100 and 150 respectively is 1. Therefore, the computer device can determine the pixel color enhancement coefficient of the pixel point according to the color enhancement mapping relationship stored in the mapping table and the pixel color value.

[0103] The color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area.

[0104] In order to avoid color discontinuity and jumps in the color values ​​in the color protection area and the non-color protection area after color enhancement, the color enhancement coefficient corresponding to the color value in the transition area is set to between the color enhancement coefficients corresponding to the color values ​​in the color protection area and the non-color protection area. The color enhancement effect in the color protection area can be gradually transitioned to the color enhancement effect in the non-color protection area instead of abruptly cutting and changing. Therefore, color discontinuity and jumps can be avoided, thereby improving the quality of the color enhanced image.

[0105] For example only, Figure 4 In the smaller irregular area 430 where the color protection area is located, the color enhancement coefficient is 1, and in the area within the chromaticity area 420 where the non-color protection area is located and outside the larger irregular area 440, the color enhancement coefficient is 1.35.

[0106] In one possible scenario, the color enhancement coefficient in the transition area (such as an area located within the larger irregular area 440 and outside the smaller irregular area 430) is a fixed value (such as 1.2) between the color enhancement coefficient corresponding to the color protection area (such as 1) and the color enhancement coefficient corresponding to the non-color protection area (such as 1.35).

[0107] In another possible scenario, the color enhancement coefficient in the transition area is a continuously changing value (such as a continuously changing value between 1 and 1.35) between the color enhancement coefficient corresponding to the color protection area (such as 1) and the color enhancement coefficient corresponding to the non-color protection area (such as 1.35).

[0108] Step 203 , according to the pixel color enhancement coefficient of each pixel point, the color of each pixel point in the image is enhanced to obtain a color enhanced image.

[0109] Optionally, color enhancement includes but is not limited to one or more color enhancement methods such as contrast enhancement, saturation enhancement, brightness adjustment, white balance adjustment, etc.

[0110] Regarding the specific method of enhancing the color of each pixel in an image according to the pixel color enhancement coefficient, in one possible implementation method, a computer device can determine the product of the pixel color value corresponding to each pixel in the image in the color space and the pixel color enhancement coefficient as the pixel enhanced color value corresponding to the pixel; and then convert the pixel enhanced color value corresponding to each pixel into the RGB color space to obtain a color enhanced image.

[0111] Regarding the specific method of enhancing the color of each pixel in an image according to the pixel color enhancement coefficient, in another possible implementation method, the computer device can also determine the initial color enhanced image corresponding to the image through a lookup table corresponding to the color enhancement; and based on the pixel color values ​​corresponding to each pixel in the image frame in the color space, determine the coefficient image composed of the pixel color enhancement coefficients corresponding to each pixel; finally, merge the coefficient image with the initial color enhanced image to obtain the color enhanced image.

[0112] For more details about color enhancement of each pixel in the image according to the pixel color enhancement coefficient, please refer to the following embodiments and related descriptions, which will not be repeated here.

[0113] To summarize, the color space includes a color protection area, a non-color protection area and a transition area. Since the transition area is located between the color protection area and the non-color protection area, the color value to be protected has a high probability of falling into the color protection area, a low probability of falling into the transition area, and a very low probability of falling into the non-color protection area, thereby avoiding color omissions. At the same time, since the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area, after color enhancement, the color enhancement effect of the color protection area can be gradually transformed into the color enhancement effect of the non-color protection area in a transitional manner, rather than abruptly split and changed, thereby avoiding color discontinuity and jumps, thereby improving the quality of the color enhanced image.

[0114] In some embodiments, the computer device divides the color space into a color protection area, a transition area, and a non-color protection area.

[0115] In a possible scenario, the color space is a YUV color space, and the computer device divides the UV plane in the YUV color space into a color protection area, a transition area, and a non-color protection area, wherein the UV plane is a plane with a color value U as a horizontal coordinate and a color value V as a vertical coordinate.

[0116] In some embodiments, the computer device determines a first shape region in a UV plane of a YUV color space based on a chromaticity range corresponding to a color value to be protected.

[0117] Optionally, the shape of the first shape area may be any possible shape such as a rectangle, a circle, a triangle, a polygon or an irregular shape. In this embodiment, the first shape area is described as a first rectangular area as an example, but this is not limited to this.

[0118] The color value to be protected may be preset by the computer device. Optionally, when the color space is a YUV color space, the chromaticity range corresponding to the color value to be protected may include a color threshold corresponding to the horizontal and vertical coordinates. The color threshold may be preset based on an empirical value.

[0119] For example only, the chromaticity range corresponding to the skin color value to be protected includes color thresholds 73 and 128 corresponding to the abscissa U, and color thresholds 134 and 186 corresponding to the ordinate V.

[0120] In some embodiments, the computer device expands the first shape region in the UV plane according to the first proportionality coefficient to obtain the second shape region.

[0121] For example only, when the first shape area is a first rectangular area, the first proportional coefficient includes a length proportional coefficient (e.g. 1.5) and the width scale factor (such as 1.4). Computer equipment in the UV plane according to the length scale factor Expand the length of the first rectangular area and the width ratio coefficient The width of the first rectangular area is expanded to obtain a second shape area.

[0122] The specific method for determining the first proportionality coefficient may include at least one of the following.

[0123] (1) Determine a first scaling factor according to the image content type of the image.

[0124] Different image content types correspond to different first proportionality coefficients.

[0125] For example, in the color enhancement scenario of skin color protection, when the image content type is a portrait type, the first proportional coefficient is 1.6; when the image content type is a landscape type, the first proportional coefficient is 1.2. That is, when the image content type is a portrait type, by setting a higher first proportional coefficient, the area of ​​the transition area can be relatively larger, thereby further increasing the probability that the pixel color value corresponding to the skin color pixel falls within the skin color area and the transition area, further avoiding color leakage.

[0126] In some embodiments, when the image is a video frame in a video, the first scaling factor may also be determined according to the video type.

[0127] For example, in the color enhancement scenario of skin color protection, when the video type is a TV series, the first proportional coefficient is 1.7; when the image content type is a documentary type, the first proportional coefficient is 1.2. That is, when the video type is a TV series, by setting a higher first proportional coefficient, the area of ​​the transition area can be relatively larger, thereby further increasing the probability that the pixel color value corresponding to the skin color pixel falls within the skin color area and the transition area, further avoiding color leakage.

[0128] (2) Determine the first proportionality coefficient according to the area ratio of the ROI region in the image.

[0129] The first proportionality coefficient is positively correlated to the area ratio, and the pixel color value corresponding to the pixel point in the ROI area is located in the color protection area.

[0130] Optionally, the ROI region may be a region of interest identified by a computer device based on a pre-trained image detection model.

[0131] For example only, in a color enhancement scenario of skin color protection, the ROI area is a skin color area (eg, a face, bare arms, neck, etc. in an image).

[0132] For example only, in the color enhancement scenario of food protection, the ROI area is an area formed by the pixel points corresponding to the position of the food in the image.

[0133] For example, when the ROI area is a skin color area, and the area of ​​the ROI area in the image accounts for 10%, the first proportional coefficient is 1.3; when the area of ​​the ROI area in the image accounts for 20%, the first proportional coefficient is 1.5. That is, when the area of ​​the ROI area accounts for a large proportion, by setting a higher first proportional coefficient, the area of ​​the transition area can be relatively larger, thereby further increasing the probability that the pixel color value corresponding to the skin color pixel falls within the skin color area and the transition area, and further avoiding color missed detection.

[0134] See also Figure 5 , Figure 5 It is a schematic diagram of determining a first rectangular area and a second rectangular area provided by an exemplary embodiment of the present application.

[0135] According to the experience gained from experiments, the skin color area in the UV plane is the area that satisfies Formula 2.

[0136] ;(Formula 2)

[0137] like Figure 5 As shown, the region with color in the UV plane 510 is the chromaticity region 520. For the convenience of representation, the representation method of the rectangle in the UV plane 510 is defined as , that is, the coordinate point of the upper left corner of the rectangle ( ) and the lower right corner coordinate point . Then the skin color area is the first rectangular area 530, that is The first rectangular area 530 has a center coordinate of (100.5, 160), a width of 55, and a height of 52.

[0138] In some embodiments, the computer device expands the first shape region in the UV plane according to a first proportionality coefficient (including a length proportionality coefficient and a width proportionality coefficient) to obtain a second shape region.

[0139] For example only, the computer device takes the center point (100.5, 160) of the first rectangular area 530 as the center, and expands the width and height of the first rectangular area 530 in proportion. , times ( , are both greater than 1), obtaining a second rectangular area 540 surrounding the first rectangular area 530, and recording the first rectangular area 530 as , the second rectangular area 540 is The interior of the second rectangular area 540 and the exterior of the first rectangular area 530 are transition areas. Figure 5 As shown. Represents a rectangle external, represents the union of two regions (i.e., the common region), then the transition region is expressed as .

[0140] In some embodiments, the computer device determines a chromaticity region in the UV plane that is within the first shape region as a color protection region; determines a chromaticity region in the UV plane that is within the second shape region and outside the first shape region as a transition region; and determines a chromaticity region in the UV plane that is outside the second shape region as a non-color protection region.

[0141] like Figure 5 In the embodiment, the computer device determines the chromaticity area within the first rectangular area 530 in the UV plane as the skin color area; determines the chromaticity area within the second rectangular area 540 in the UV plane and outside the first rectangular area 530 as the transition area; and determines the chromaticity area outside the second rectangular area 540 in the UV plane as the non-skin color area.

[0142] exist and When the value is larger, the transition area is larger, but more color values ​​other than the color value to be protected are also protected, reducing the color enhancement effect; and If the value is too small, it is difficult to ensure the smoothness and naturalness of the color transition. and About 1.5, the coordinates of the upper left corner of the transition area are (60, 120), and the coordinates of the lower right corner are (140, 200), that is, .

[0143] After dividing the color space into a color protection area, a transition area, and a non-color protection area, the computer device may first determine a first color enhancement coefficient corresponding to a color value in the color protection area and a second color enhancement coefficient corresponding to a color value in the non-color protection area.

[0144] In a possible implementation, the first color enhancement coefficient and the second color enhancement coefficient are values ​​preset by the computer device. For example, in a color enhancement scenario of skin color protection, the first color enhancement coefficient is 1, and the second color enhancement coefficient is 1.3.

[0145] In a possible implementation manner, the computer device may also determine the first color enhancement coefficient according to the image content type of the image.

[0146] Different image content types correspond to different first color enhancement coefficients.

[0147] For example only, in the color enhancement scenario of skin color protection, when the image content type is a portrait type, the first color enhancement coefficient is 1. When the image content type is a landscape type, the first color enhancement coefficient is 1.3. That is, when the image content type is a portrait type, by setting the first color enhancement coefficient to a smaller value, it is possible to ensure better skin color protection for face type images, thereby improving the quality of color enhanced images.

[0148] After determining the first color enhancement coefficient and the second color enhancement coefficient, in some embodiments, the computer device determines a third color enhancement coefficient corresponding to the color value in the transition area based on the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area.

[0149] The difference between the third color enhancement coefficient and the first color enhancement coefficient is positively correlated with the distance, and the difference between the second color enhancement coefficient and the third color enhancement coefficient is negatively correlated with the distance.

[0150] That is, the third color enhancement coefficient corresponding to the color value in the transition area changes continuously. When the color value in the transition area is close to the color protection area, the third color enhancement coefficient is close to the first color enhancement coefficient; when the color value in the transition area is close to the non-color protection area, the third color enhancement coefficient is close to the second color enhancement coefficient.

[0151] In one possible manner, at least one of the difference between the third color enhancement coefficient and the first color enhancement coefficient, or the difference between the second color enhancement coefficient and the third color enhancement coefficient, may be linearly correlated or nonlinearly correlated with the distance from the color value in the transition area to the color protection area, and the present application does not limit the specific form of the correlation.

[0152] In a possible implementation manner, the computer device determines the interpolation coefficient based on the distance from the color value in the transition area to the color protection area.

[0153] In one possible manner, the computer device performs distance normalization processing on each distance from each color value in the transition area to the color protection area to obtain an interpolation coefficient. For example, the normalized distance is obtained by a method such as Min-Max Normalization or Mean Normalization, and the normalized distance is used as the interpolation coefficient.

[0154] In some embodiments, the computer device interpolates the first color enhancement coefficient and the second color enhancement coefficient according to the interpolation coefficient to obtain a third color enhancement coefficient corresponding to the color value in the transition area.

[0155] For example only, the second color enhancement coefficient is , the first color enhancement coefficient is , then the third color enhancement coefficient can be calculated by the following formula.

[0156] The third color enhancement coefficient = interpolation coefficient × ( - )+ .

[0157] In some embodiments, when the transition area is a chromaticity area located outside the first rectangular area and within the second rectangular area in the UV plane of the YUV color space, in order to ensure that the color enhancement coefficients on both sides of the inner and outer sides of the points on the outer boundary of the transition area are continuous, the computer device can also correct the transition area so that the transition area becomes a strip area of ​​equal width.

[0158] In some embodiments, the computer device sets the second rectangular area The color values ​​on the color protection area Distance , determined as a distance set.

[0159] Optionally, the computer device calculates the second rectangular area using cv2.distanceTransform in the image processing library OpenCV. The color values ​​on the color protection area Distance .

[0160] In some embodiments, the computer device sets the minimum distance in the distance set Determined as the distance threshold .Right now, .

[0161] In some embodiments, the computer device modifies the transition region , where the corrected transition region Color values ​​in the color protection area The distance is less than the distance threshold .

[0162] For example, for the upper left corner vertex (60, 120) of the second rectangular area 540, the upper left corner vertex (60, 120) to the color protection area The distance is greater than the distance threshold , so the corrected transition region The vertex (60, 120) is not included in the result. Similarly, other vertices are modified similarly.

[0163] See also Figure 6 , Figure 6 is a schematic diagram of a modified transition region provided by an exemplary embodiment of the present application.

[0164] like Figure 6 As shown, the modified transition area is a chromaticity area located outside the first rectangular area 610 and within the modified rounded area 620. Since the modified transition area becomes a strip area of ​​equal width (that is, the distance from the modified transition area to the color protection area is less than the distance threshold), the color enhancement coefficients on both sides of the inner and outer sides of the points on the outer boundary of the transition area can be continuous, and a smooth transition of colors can be achieved during color enhancement, thereby improving the quality of the color enhanced image.

[0165] In some embodiments, after the transition area is corrected, in a possible implementation, the computer device determines the ratio of the distance from the color value in the transition area to the color protection area to the distance threshold as the interpolation coefficient.

[0166] For example only, the corrected transition area Every point in If it reaches the skin color area The distance is , then the interpolation coefficient can be expressed as .

[0167] For example only, the second color enhancement coefficient is , the first color enhancement coefficient is , then the third color enhancement coefficient can be calculated by the following formula.

[0168] Third color enhancement factor = ×( - )+ .

[0169] See also Figure 7 , Figure 7It is a schematic diagram of the distribution of color enhancement coefficients provided by an exemplary embodiment of the present application.

[0170] like Figure 7 As shown in (a), black represents the first color enhancement coefficient (1), white represents the second color enhancement coefficient (1.35), and grayscale represents the second color enhancement coefficient (1~1.35). It can be seen that the color enhancement coefficient in the color protection area (such as the skin color area) in the UV plane is the first color enhancement coefficient, the color enhancement coefficient in the non-color protection area (such as the non-skin color area) is the second color enhancement coefficient, and the third color enhancement coefficient in the transition area between the two areas (such as the near-skin color area) realizes a gradient from 1 to 1.35.

[0171] Figure 7 (b) in the figure is the color enhancement coefficient change curve along V=160, as shown in Figure 7 As shown in (b), the third color enhancement coefficient located in the transition area between the color protection area and the non-color protection area realizes a gradual change from 1 to 1.35.

[0172] Regarding the specific method of dividing the color space into a color protection area, a transition area, and a non-color protection area, in some embodiments, the computer device determines the inscribed ellipse of the chromaticity area in the UV plane of the YUV color space as the first ellipse.

[0173] Optionally, a specific method for determining the first ellipse includes the following steps.

[0174] S1: Create a single channel image M of size 256x256.

[0175] S2: traverse and calculate the corresponding U and V values ​​when R, G, and B are 0 to 255 respectively, and take U and V as pixel coordinate values, and set M [U, V] = 255. After the traversal is completed, the pixel value of the area with color distribution in the single-channel image M is 255, and the pixel value of the area without color definition is 0.

[0176] S3: Use the image processing library OpenCV to obtain the maximum connected domain in the image M.

[0177] A connected domain is defined as a set of pixels with the same pixel value and adjacent in space. Use the function cv2.connectedComponents to mark each connected domain, and get the number of connected domains and the serial number corresponding to each connected domain. Use the function cv2.connectedComponentsWithStats to get the area of ​​each connected domain. The connected domain with the largest area in the single-channel image M is the area with color distribution.

[0178] S4: Using the image processing library OpenCV, obtain the inscribed ellipse of the largest connected domain in the image M, that is, the first ellipse.

[0179] An inscribed ellipse is an ellipse that is tangent to the maximum connected domain and is completely enclosed within the maximum connected domain.

[0180] Use the function cv2.fitEllipse to obtain the center point of the inscribed ellipse in the largest connected domain (should be the coordinate point (128, 128)), the length of the major axis , short axis length And the ellipse rotation angle .

[0181] See also Figure 8 , Figure 8 It is a schematic diagram of a first ellipse provided by an exemplary embodiment of the present application.

[0182] like Figure 8 As shown, the inscribed ellipse of the chromaticity region 820 of the UV plane 810 is the first ellipse 830. For ease of representation, the representation method of the ellipse in the UV plane is defined as , that is, the coordinates of the ellipse origin , major axis length , short axis length , the ellipse rotation angle Then the inscribed ellipse (first ellipse) of the chromaticity region in the UV plane is ,in The distribution is the minor axis and major axis of the first ellipse.

[0183] In some embodiments, the computer device shrinks the first ellipse in the UV plane according to a second proportionality factor to obtain a second ellipse.

[0184] In a possible implementation, the second proportionality coefficient is a value preset by the computer device, such as 0.8.

[0185] In a possible implementation manner, the computer device determines the second proportional coefficient based on the transition factor and the second color enhancement coefficient.

[0186] The second proportionality coefficient is positively correlated to the transition factor, and negatively correlated to the second color enhancement coefficient.

[0187] For example only, the second proportionality factor is , the second color enhancement coefficient is , the transition factor is , then the second proportionality coefficient is It can be expressed as .

[0188] In some embodiments, color enhancement includes saturation enhancement. After color enhancement, the color value moves in a direction away from the center point (128, 128). To ensure that the color value within the second ellipse does not exceed the first ellipse after color enhancement, when the second color enhancement coefficient in the non-color protection area is large, the second proportional coefficient is set to be small, so that the second ellipse can be made smaller, thereby ensuring that the color value within the second ellipse does not exceed the first ellipse after color enhancement.

[0189] See also Fig. 9 , Fig. 9 is a schematic diagram of a second ellipse provided by an exemplary embodiment of the present application.

[0190] like Fig. 9 As shown, in the UV plane, the major and minor axes of the first ellipse 910 are proportional to Contraction, with the coordinate point (128, 128) as the origin of the ellipse, , are the lengths of the minor and major axes of the ellipse, is the ellipse rotation angle, and the second ellipse 920 is recorded as .

[0191] In some embodiments, the computer device determines the chromaticity area in the UV plane that is outside the first ellipse as a color protection area (the difference between the chromaticity area 930 and the first ellipse 910); determines the chromaticity area in the UV plane that is inside the first ellipse and outside the second ellipse as a transition area (the difference between the first ellipse 910 and the second ellipse 920); and determines the chromaticity area in the UV plane that is inside the second ellipse as a non-color protection area (the area inside the second ellipse 920).

[0192] Optionally, the color protection area is a high-saturation area with a saturation greater than a first threshold, the non-color protection area is a low-saturation area with a saturation less than a second threshold, and the transition area is a near-high-saturation area with a saturation greater than the second threshold and less than the first threshold.

[0193] The transition factor is used to control the area distribution of the transition area before and after color enhancement.

[0194] See also Fig.10 , Fig.10 It is a schematic diagram of the regional distribution of the transition area before and after saturation enhancement provided by an exemplary embodiment of the present application.

[0195] like Fig.10 As shown, before the saturation is enhanced, the short axis length of the first ellipse 1011 is , using the second proportionality factor The short axis length of the second ellipse 1021 obtained by contraction is .

[0196] The geometric meaning of saturation enhancement in the UV plane can be understood as the pixel moving away from the center point (128, 128). Fig.10 As shown, the first ellipse 1011 ( ) is the outer boundary of the high saturation area, and the short axis and long axis are , the second ellipse 1021 ( ) is the boundary of the near high saturation region, and the short axis and long axis are The second color enhancement coefficient near the boundary of the high saturation region is , the points on the boundary after color enhancement fall on the major and minor axes respectively On the concentric ellipse, the third ellipse 1022 is recorded as , then after the color is enhanced, the original second ellipse The inner color will be distributed to the third ellipse and due to the first color enhancement factor , the first ellipse To ensure the monotony of color enhancement, the area After color enhancement, it should be distributed in the area As mentioned earlier, ,therefore , that is, the transition factor Used to control the area distribution of the transition area (near high saturation area) before and after color enhancement.

[0197] In some embodiments, color enhancement includes saturation enhancement, and the computer device may determine a transition factor based on the saturation of the image, wherein the saturation is negatively correlated to the transition factor.

[0198] For example, when the image has a high saturation, the second proportional coefficient can be further reduced by selecting a smaller transition factor, thereby making the second ellipse smaller, further ensuring that the color value saturation inside the second ellipse will not exceed the first ellipse after being enhanced.

[0199] See also Fig.11 , Fig.11 This is a flowchart of determining a third color enhancement coefficient corresponding to a color value in a transition area according to a first color enhancement coefficient, a second color enhancement coefficient, and a distance from a color value in a transition area to a color protection area provided by an exemplary embodiment of the present application. The flow includes the following steps.

[0200] Step 1101, determine the first circle with the minor axis of the first ellipse as the radius and the point with the lowest saturation in the UV plane as the center; determine the second circle with the minor axis of the second ellipse as the radius and the point with the lowest saturation in the UV plane as the center.

[0201] For example, with the point (128, 128) with the lowest saturation in the UV plane as the center, and Draw two concentric circles for the radius and .

[0202] Step 1102: determine the chromaticity region in the UV plane that is within the first circle and outside the second circle as the annular region.

[0203] For example, two concentric circles and The annular zone between .

[0204] In some embodiments, the computer device calculates the annulus area The distance from each point on the circle to the center (128, 128) is normalized to 0-1.

[0205] Step 1103, determining the affine transformation relationship between the annular zone and the transition zone.

[0206] The method of mapping the annular region to the transition region (such as the near high saturation region) is affine transformation. Affine transformation transforms a vector space into another vector space by linear transformation and translation. In image processing, it can be understood as scaling, rotating and translating the image. The coordinates are , remember the points in the transition region (such as the near high saturation region) The coordinates are , the mapping process can be expressed as the following formula.

[0207] ;

[0208] Where A is the rotation and scaling matrix, and b is the translation vector.

[0209] , ;

[0210] Affine transformation requires a total of 6 parameters, and solving A and b requires at least three pairs of matching points (each matching point can provide two equations). Therefore, three matching points are taken in the transition area (such as the near high saturation area) and the annulus area, and the OpenCV function getAffineTransform is used to calculate the affine transformation parameters A and b from the circular annulus to the near high saturation area.

[0211] See also Fig.12 , Fig.12 It is a schematic diagram of an affine transformation relationship provided by an exemplary embodiment of the present application.

[0212] like Fig.12 As shown, take three points in the inner and outer ellipse vertices near the high saturation area , and the matching points of the corresponding circular rings , the corresponding coordinates are expressed as the following formulas.

[0213] ;

[0214] ;

[0215] ;

[0216] ;

[0217] ;

[0218] ;

[0219] Step 1104, mapping the distance from the color value in the transition area to the color protection area to the distance from the color value in the annular area to the center of the circle, wherein the color value in the transition area is mapped to the color value in the annular area based on an affine transformation relationship.

[0220] After obtaining the affine transformation parameters, use the OpenCV affine transformation function warpAffine to perform affine transformation and convert the color values ​​(points) in the transition area ) to the color protection area, mapped to the distance from the color value in the annular area to the center of the circle, recorded as .

[0221] Step 1105 , determining a third color enhancement coefficient corresponding to the color value in the transition area according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the annular area to the center of the circle.

[0222] In the case where color enhancement includes saturation enhancement, the third color enhancement coefficient of the transition region (ie, the near high saturation region) needs to meet the following conditions.

[0223] (1) The third color enhancement coefficient should achieve the second color enhancement coefficient To the first color enhancement factor Gradual transition, that is, the closer to the high saturation area, the closer the third color enhancement coefficient is .

[0224] (2) The third color enhancement coefficient should ensure the monotonicity of the enhancement. Assume that the saturation of the first two pixels A and B is enhanced. , satisfy , then the saturation should still meet the color enhancement .

[0225] In some embodiments, the computer device interpolates the first color enhancement coefficient and the second color enhancement coefficient according to the interpolation coefficient to obtain a third color enhancement coefficient corresponding to the color value in the transition area.

[0226] For example only, the second color enhancement coefficient is , the first color enhancement coefficient is , then the third color enhancement coefficient It can be calculated by the following formula.

[0227] The third color enhancement factor = interpolation coefficient × ( - )+ .

[0228] Optionally, the computer device may also use a nonlinear interpolation method to determine the third color enhancement coefficient corresponding to the color value in the transition area. .

[0229] For example only, the third color enhancement factor It can be expressed as the following formula.

[0230] ;

[0231] in, is the second color enhancement coefficient, is the first color enhancement coefficient, is the transition factor, is the color value in the transition area (point ) to the color protection area, mapped to the distance from the color value in the annular area to the center of the circle. It can be seen that the third color enhancement coefficient Achieved distance The increase of Gradually .

[0232] See also Fig.13 , Fig.13 It is a schematic diagram of the distribution of the third color enhancement coefficient provided by an exemplary embodiment of the present application.

[0233] like Fig.13 As shown in (a), black represents the first color enhancement coefficient (1), white represents the second color enhancement coefficient (1.35), and grayscale represents the second color enhancement coefficient (1~1.35). It can be seen that the color enhancement coefficient in the color protection area (such as the high saturation area) in the UV plane is the first color enhancement coefficient, the color enhancement coefficient in the non-color protection area (such as the low saturation area) is the second color enhancement coefficient, and the third color enhancement coefficient in the transition area between the two areas (such as the near high saturation area) realizes a gradient from 1 to 1.35.

[0234] Fig.13 (b) in the figure is the color enhancement coefficient change curve along V=128, as shown in Fig.13 As shown in (b), the third color enhancement coefficient located in the transition area between the color protection area and the non-color protection area realizes a nonlinear gradient from 1 to 1.35.

[0235] The above embodiment introduces the color enhancement scenario under skin color protection by taking the color protection area as the skin color area, the transition area as the near skin color area, and the non-color protection area as the non-skin color area as an example. It also introduces the color enhancement scenario under saturation protection by taking the color protection area as the high saturation area, the transition area as the near high saturation area, and the non-color protection area as the low saturation area as an example.

[0236] In the color enhancement scene under skin color protection, skin color pixels and high saturation pixels can also be protected at the same time.

[0237] In some embodiments, the color protection area includes a skin color area and a high saturation area, the transition area includes a near skin color area and a near high saturation area, and the non-color protection area is a chromaticity area outside the color protection area and the transition area in the color space.

[0238] In one possible case, there may be an overlapping area between the near high saturation area and the skin color area or the near skin color area.

[0239] In one possible scenario, when the pixel color value is located in the overlapping area of ​​the near-high saturation area and the skin color area, the computer device determines the third color enhancement coefficient corresponding to the pixel color value in the near-high saturation area and the first color enhancement coefficient corresponding to the skin color area based on the color enhancement mapping relationship.

[0240] For example only, the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area is 1.2, and the first color enhancement coefficient corresponding to the pixel color value in the skin color area is 1.

[0241] Optionally, the computer device determines the minimum value between the first color enhancement coefficient and the third color enhancement coefficient as the pixel color enhancement coefficient of the pixel point.

[0242] For example, the first color enhancement coefficient 1 corresponding to the pixel color value in the skin color area is less than the third color enhancement coefficient 1.2 corresponding to the near high saturation area, so the minimum value 1 is determined as the pixel color enhancement coefficient of the pixel point.

[0243] Optionally, the computer device determines a fusion value obtained by fusing the first color enhancement coefficient and the third color enhancement coefficient as the pixel color enhancement coefficient of the pixel point.

[0244] The fusion value may be the product of the first color enhancement coefficient and the third color enhancement coefficient, or may be a result obtained by fusion in any other possible manner, such as weighted fusion, etc., and there is no limitation on this.

[0245] For example, the first color enhancement coefficient 1 corresponding to the pixel color value in the skin color area is less than the third color enhancement coefficient 1.2 corresponding to the near high saturation area, so the product 1.2 is determined as the pixel color enhancement coefficient of the pixel point.

[0246] In one possible scenario, when the pixel color value is located in the overlapping area of ​​the near high saturation area and the near skin color area, the computer device determines the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area based on the color enhancement mapping relationship.

[0247] For example only, the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area is 1.2, and the third color enhancement coefficient corresponding to the near skin color area is 1.1.

[0248] Optionally, the computer device determines the minimum value of the third color enhancement coefficient corresponding to the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area as the pixel color enhancement coefficient of the pixel point.

[0249] For example, the third color enhancement coefficient 1.1 corresponding to the pixel color value in the near skin color area is smaller than the third color enhancement coefficient 1.2 corresponding to the near high saturation area, so the minimum value 1.1 is determined as the pixel color enhancement coefficient of the pixel point.

[0250] Optionally, the computer device determines a fusion value of a third color enhancement coefficient corresponding to a near high saturation area and a third color enhancement coefficient corresponding to a near skin color area as the pixel color enhancement coefficient of the pixel point.

[0251] The fusion value may be a product, or a result obtained by fusion in any other possible manner, such as weighted fusion, etc., and there is no limitation on this.

[0252] For example, the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area is 1.2, and the third color enhancement coefficient corresponding to the near skin color area is 1.1, so the product 1.32 is determined as the pixel color enhancement coefficient of the pixel point.

[0253] See also Fig.14 , Fig.14 It is a schematic diagram of the distribution of color enhancement coefficients when protecting skin color pixels and high-saturation pixels at the same time, provided by an exemplary embodiment of the present application.

[0254] like Fig.14 As shown, black represents the first color enhancement coefficient (1), white represents the second color enhancement coefficient (1.35), and grayscale represents the second color enhancement coefficient (1~1.35). It can be seen that the color enhancement coefficient in the color protection area (including the high saturation area and the skin color area) in the UV plane is the first color enhancement coefficient, the color enhancement coefficient in the non-color protection area (the chromaticity area outside the color protection area and the transition area) is the second color enhancement coefficient, and the third color enhancement coefficient in the transition area between the two areas (including the near high saturation area and the near skin color area) realizes a gradient from 1 to 1.35.

[0255] After determining the color enhancement coefficients corresponding to the color values ​​in the color protection area, transition area and non-color protection area, the computer device can determine the pixel color enhancement coefficient of the pixel point based on the color enhancement mapping relationship and the pixel color value, and enhance the color of each pixel point in the image according to the pixel color enhancement coefficient to obtain a color enhanced image.

[0256] There may be various methods for obtaining a color enhanced image. The following uses a saturation linear stretch enhancement algorithm and a color enhancement algorithm based on a three-dimensional lookup table (3DLUT) as examples. For other color enhancement algorithms, practitioners in this field and industry can easily draw inferences from the example algorithms.

[0257] (1) Saturation linear stretch enhancement algorithm.

[0258] Traditional saturation linear stretch enhancement is to linearly stretch the U and V channels of the image to achieve saturation changes. , the color enhancement factor is , then the U and V values ​​after saturation enhancement are . Let the input image be , RGB are the pixel values ​​of the red, green, and blue channels of the image respectively; color enhancement mapping relationship It is a lookup table of size 256x256, which stores the color enhancement coefficients corresponding to the pixel U and V values ​​changing from 0 to 255, such as Indicates that the input pixel U and V values ​​are When , the corresponding preset color enhancement coefficient is .

[0259] In some embodiments, the computer device determines the product of the pixel color value corresponding to each pixel point in the image in the YUV color space and the pixel color enhancement coefficient as the pixel enhanced color value corresponding to the pixel point.

[0260] For example, the input image Convert to YUV color space, pixels , in the color enhancement mapping relationship Read the corresponding color enhancement coefficient ; The image Every pixel in The U and V values ​​are updated to .

[0261] In some embodiments, the computer device converts the pixel-enhanced color value corresponding to each pixel point into the RGB color space to obtain a color-enhanced image.

[0262] Exemplarily, the computer device converts the pixel enhanced color value from the YUV color space to the RGB color space based on Formula 1 to obtain a color enhanced image.

[0263] (2) Color enhancement algorithm based on three-dimensional lookup table (3DLUT).

[0264] In some embodiments, the computer device determines an initial color-enhanced image corresponding to the image through a lookup table corresponding to the color enhancement and the RGB value of each pixel in the image.

[0265] The input image is recorded as , RGB are the red, green, and blue channel pixel values ​​of the video frame respectively; the three-dimensional lookup table LUT is a lookup table with a size of 256x256x256, which stores the corresponding output RGB values ​​when the R, G, and B values ​​change from 0 to 255, such as Indicates that the input pixel values ​​RGB are When , the output RGB values ​​after color enhancement are . For the input video frame Every pixel , look up the 3DLUT table to get the corresponding pixel output value , get the output image .

[0266] In some embodiments, the computer device determines a coefficient image composed of pixel color enhancement coefficients corresponding to each pixel point based on the pixel color value corresponding to each pixel point in the image frame in the YUV color space.

[0267] Among them, the coefficient image has the same size as the initial color enhanced image.

[0268] Exemplarily, the computer device converts the image according to Formula 1 Convert to YUV color space and Every pixel in , in the color enhancement mapping relationship Read the corresponding color enhancement coefficient , forming and image Coefficient images of color enhancement coefficients of the same size .

[0269] In some embodiments, the computer device merges the coefficient image with the initial color-enhanced image to obtain a color-enhanced image.

[0270] Exemplarily, the computer device converts the coefficient image Normalize the values ​​in to the range of 0 to 1 and update the output image .

[0271] In this embodiment, the coefficient image The color enhancement coefficient of the skin color area is small, and tends to 0 after normalization. The color enhancement coefficient of the non-skin color area is large, and tends to 1 after normalization. Therefore, the updated output image In the image, the skin color area is close to the input image , the non-skin color area is close to the image after 3DLUT enhancement By utilizing the continuity of colors in the UV plane to achieve smooth color transition, and the existence of a transition area to ensure the smoothness of colors in space, the smoothness and stability of colors in space and time in the skin color protection scene is achieved.

[0272] The image color enhancement method proposed in this application can be applied to color enhancement scenarios of video applications, live broadcast applications, photo beauty applications, or social applications.

[0273] Color enhancement scenarios in video applications

[0274] In an optional example, the image color enhancement method provided in the embodiment of the present application is applied to a color enhancement scenario in a video application as an example for introduction.

[0275] In a possible scenario, the color enhancement method of the image may be executed by a server.

[0276] Optionally, the image is a video frame extracted from a video, and the color-enhanced image is a color-enhanced video.

[0277] In some embodiments, the server obtains a video frame from the video, and uses the image color enhancement method provided in the present application to perform color enhancement processing on the video frame to obtain a color enhanced video frame.

[0278] In some embodiments, the server generates a color-enhanced video based on the color-enhanced video frames.

[0279] In some embodiments, upon receiving a color enhancement request sent by a terminal, the server transmits a color enhanced video to the terminal.

[0280] In some embodiments, the terminal plays the video before color enhancement through the video application. When the user performs the color enhancement operation through the operation instruction, the terminal sends a color enhancement request to the server to obtain the color enhanced video from the server and display it. For example, when the user clicks the color enhancement control displayed on the terminal, or triggers the shortcut key corresponding to the color enhancement, the terminal obtains the color enhanced video from the server and displays it.

[0281] See also Fig.15 , Fig.15 It is a schematic diagram of a terminal display color enhancement control provided by an exemplary embodiment of the present application.

[0282] like Fig.15 In the video playback interface displayed by the terminal, a color enhancement control 1501 is included. When a user triggers the color enhancement control 1501, the terminal obtains the color-enhanced video from the server and displays it.

[0283] The color-enhanced video is obtained by the server performing color enhancement on video frames contained in the video in advance or in real time.

[0284] In some embodiments, the server obtains pixel color values ​​of pixels in a video frame in a YUV color space, where the YUV color space includes a skin color area, a non-skin color area, and a near-skin color area between the skin color area and the non-skin color area.

[0285] In some embodiments, the pixel color enhancement coefficient of the pixel point is determined based on the color enhancement mapping relationship and the pixel color value. The color enhancement mapping relationship represents the correspondence between the color value in the YUV color space and the color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the near skin color area is between the color enhancement coefficient corresponding to the color value in the skin color area and the color enhancement coefficient corresponding to the color value in the non-skin color area.

[0286] In some embodiments, the server enhances the color of each pixel in the video frame according to the pixel color enhancement coefficient to obtain a color enhanced video frame.

[0287] Optionally, the server divides the YUV color space into a skin color area, a near skin color area, and a non-skin color area.

[0288] Optionally, the server determines a first color enhancement coefficient corresponding to the color value in the skin color area, and a second color enhancement coefficient corresponding to the color value in the non-skin color area.

[0289] Optionally, the server determines a third color enhancement coefficient corresponding to the color value in the near skin color area based on the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the near skin color area to the skin color area, wherein the difference between the third color enhancement coefficient and the first color enhancement coefficient is positively correlated with the distance, and the difference between the second color enhancement coefficient and the third color enhancement coefficient is negatively correlated with the distance.

[0290] Regarding the specific way in which the server divides the color space into skin color areas, near skin color areas, and non-skin color areas, in some embodiments, the server determines a first rectangular area in the UV plane of the YUV color space based on the chromaticity range corresponding to the color value to be protected; expands the first rectangular area in the UV plane according to a first proportional coefficient to obtain a second rectangular area; determines the chromaticity area within the first rectangular area in the UV plane as the skin color area; determines the chromaticity area within the second rectangular area in the UV plane and outside the first rectangular area as the near skin color area; and determines the chromaticity area outside the second rectangular area in the UV plane as the non-skin color area.

[0291] Optionally, the server determines the interpolation coefficient as the ratio of the distance from the color value in the near skin color area to the skin color area to a distance threshold.

[0292] Optionally, the server interpolates the first color enhancement coefficient and the second color enhancement coefficient according to the interpolation coefficient to obtain a third color enhancement coefficient corresponding to the color value in the near skin color area.

[0293] Optionally, the server determines the distance from each color value on the second rectangular area to the skin color area as a distance set; determines the minimum distance in the distance set as a distance threshold; and corrects the transition area, wherein the distance from the color value in the corrected transition area to the color protection area is less than the distance threshold.

[0294] See also Fig.16 , Fig.16 It is a flowchart of the construction phase and application phase of the color enhancement mapping relationship provided by an exemplary embodiment of the present application.

[0295] like Fig.16 As shown, in the color enhancement mapping relationship construction stage 1610, the steps performed by the server include steps 1611 to 1613.

[0296] Step 1611, dividing the area into a color protection area, a transition area and a non-color protection area.

[0297] For more information on how to divide color protection areas, transition areas, and non-color protection areas, see Figure 3 , Figure 4 , Figure 7 and related descriptions.

[0298] Step 1612, determining the color enhancement coefficient corresponding to the color value in each area.

[0299] In some embodiments, the server first determines a first color enhancement coefficient corresponding to a color value in a color protection area, and a second color enhancement coefficient corresponding to a color value in a non-color protection area.

[0300] In some embodiments, the server determines a third color enhancement coefficient corresponding to the color value in the transition area according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area.

[0301] The difference between the third color enhancement coefficient and the first color enhancement coefficient is positively correlated with the distance, and the difference between the second color enhancement coefficient and the third color enhancement coefficient is negatively correlated with the distance.

[0302] For more details on determining the color enhancement coefficient corresponding to the color value in each area, please refer to the relevant description of the above embodiments, which will not be repeated here.

[0303] Step 1613, construct a mapping table.

[0304] In a possible implementation, the color enhancement mapping relationship is stored through a mapping table; the mapping table contains the corresponding relationship between each color value (U, V) in the color space and the color enhancement coefficient, such as the color enhancement coefficient corresponding to the center point (128, 128) is 1.5, and the color enhancement coefficient corresponding to the point (100, 150) with a color value U and a color value V of 100 and 150 respectively is 1. Therefore, the computer device can determine the pixel color enhancement coefficient of the pixel point according to the color enhancement mapping relationship stored in the mapping table and the pixel color value.

[0305] In the application phase 1620 , the server performs steps including steps 1621 to 1625 .

[0306] Step 1621, input image I.

[0307] Optionally, pixels in the image I are represented by RGB values.

[0308] Step 1622, convert to UV plane.

[0309] Optionally, the server converts the pixel points in the image I from the RGB color space to the UV plane in the YUV color space based on Formula 1 to obtain the pixel color value (including the color value U and the color value V).

[0310] Step 1623, determine the pixel color enhancement coefficient.

[0311] In some embodiments, the server determines the pixel color enhancement coefficient of the pixel point according to the color enhancement mapping relationship and the pixel color value by looking up a table.

[0312] The color enhancement mapping relationship represents the corresponding relationship between the color value in the color space and the color enhancement coefficient, and is stored in a mapping table.

[0313] Step 1624, color enhancement.

[0314] In some embodiments, the server may enhance the color of pixels in the image I in any possible manner. For details, refer to the saturation linear stretch enhancement algorithm and the color enhancement algorithm based on a three-dimensional lookup table (3DLUT) mentioned above, which will not be expanded here.

[0315] Step 1625, obtaining a color enhanced image.

[0316] After the server enhances the color of each pixel in the image according to the pixel color enhancement coefficient corresponding to each pixel, a color enhanced image can be obtained.

[0317] See also Fig.17 , Fig.17: is a structural block diagram of an image color enhancement device provided by an exemplary embodiment of the present application. The device includes:

[0318] An acquisition module 1701 is used to acquire a pixel color value of each pixel in an image in a color space, wherein the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area;

[0319] A determination module 1702 is used to determine the pixel color enhancement coefficient of each pixel point according to the color enhancement mapping relationship and the pixel color value, wherein the color enhancement mapping relationship represents the correspondence between the color value in the color space and the color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area;

[0320] The color enhancement module 1703 is used to enhance the color of each pixel in the image according to the pixel color enhancement coefficient of each pixel to obtain a color enhanced image.

[0321] Optionally, the device further includes a region division module, which is used to:

[0322] Dividing the color space into the color protection area, the transition area and the non-color protection area;

[0323] Determine a first color enhancement coefficient corresponding to the color value in the color protection area, and a second color enhancement coefficient corresponding to the color value in the non-color protection area;

[0324] A third color enhancement coefficient corresponding to the color value in the transition area is determined based on the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area, wherein the difference between the third color enhancement coefficient and the first color enhancement coefficient is positively correlated with the distance, and the difference between the second color enhancement coefficient and the third color enhancement coefficient is negatively correlated with the distance.

[0325] Optionally, the color space is a YUV color space, and the region division module is used to:

[0326] Determining a first shape area in a UV plane of the YUV color space based on a chromaticity range corresponding to a color value to be protected;

[0327] Expand the first shape region on the UV plane according to a first proportional coefficient to obtain a second shape region;

[0328] The chromaticity area in the UV plane that is located within the first shape area is determined as the color protection area; the chromaticity area in the UV plane that is located within the second shape area and outside the first shape area is determined as the transition area; and the chromaticity area in the UV plane that is located outside the second shape area is determined as the non-color protection area.

[0329] Optionally, the area division module is used for at least one of the following:

[0330] Determining the first proportionality coefficient according to the image content type of the image, where different image content types correspond to different first proportionality coefficients;

[0331] The first proportionality coefficient is determined according to the area ratio of the ROI region in the image, the first proportionality coefficient is positively correlated to the area ratio, and the pixel color value corresponding to the pixel point in the ROI region is located in the color protection area.

[0332] Optional, zone partitioning module for:

[0333] The first color enhancement coefficient is determined according to the image content type of the image, and different image content types correspond to different first color enhancement coefficients.

[0334] Optional, zone partitioning module for:

[0335] determining an interpolation coefficient based on the distance from the color value in the transition area to the color protection area;

[0336] The first color enhancement coefficient and the second color enhancement coefficient are interpolated according to the interpolation coefficient to obtain the third color enhancement coefficient corresponding to the color value in the transition area.

[0337] Optionally, the transition area is a chromaticity area in a UV plane of a YUV color space, which is outside the first rectangular area and within the second rectangular area, and the area division module is used to:

[0338] Determine the distances from each color value on the second rectangular area to the color protection area as a distance set;

[0339] Determine the minimum distance in the distance set as a distance threshold;

[0340] Determine the ratio of the distance from the color value in the transition area to the color protection area to the distance threshold as an interpolation coefficient;

[0341] The transition area is corrected, wherein the distance from the color value in the corrected transition area to the color protection area is less than the distance threshold.

[0342] Optionally, the color protection area is a skin color area, the non-color protection area is a non-skin color area, and the transition area is a near-skin color area between the skin color area and the non-skin color area.

[0343] Optionally, the color space is a YUV color space, and the region division module is used to:

[0344] Determine an inscribed ellipse of a chromaticity region in a UV plane of the YUV color space as a first ellipse;

[0345] Contracting the first ellipse in the UV plane according to a second proportional coefficient to obtain a second ellipse;

[0346] The chromaticity area in the UV plane that is outside the first ellipse is determined as the color protection area; the chromaticity area in the UV plane that is within the first ellipse and outside the second ellipse is determined as the transition area; and the chromaticity area in the UV plane that is within the second ellipse is determined as the non-color protection area.

[0347] Optional, zone partitioning module for:

[0348] Based on the transition factor and the second color enhancement coefficient, the second proportional coefficient is determined, the second proportional coefficient is positively correlated to the transition factor and negatively correlated to the second color enhancement coefficient, and the transition factor is used to control the regional distribution of the transition area before and after the color enhancement.

[0349] Optionally, the color enhancement includes saturation enhancement, and a region division module is used to:

[0350] The transition factor is determined based on the saturation of the image, and the saturation is negatively correlated to the transition factor.

[0351] Optional, zone partitioning module for:

[0352] A first circle is determined by taking the minor axis of the first ellipse as a radius and the point with the lowest saturation in the UV plane as a center; a second circle is determined by taking the minor axis of the second ellipse as a radius and the point with the lowest saturation in the UV plane as a center;

[0353] Determine a chromaticity region in the UV plane that is within the first circle and outside the second circle as a ring zone region;

[0354] Determining an affine transformation relationship between the annular zone and the transition zone;

[0355] Mapping the distance from the color value in the transition area to the color protection area to the distance from the color value in the annular area to the center of the circle, wherein the color value in the transition area is mapped to the color value in the annular area based on the affine transformation relationship;

[0356] The third color enhancement coefficient corresponding to the color value in the transition area is determined according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the annular area to the center of the circle.

[0357] Optionally, the color protection area is a high-saturation area with a saturation greater than a first threshold, the non-color protection area is a low-saturation area with a saturation less than a second threshold, and the transition area is a near-high-saturation area with a saturation greater than the second threshold and less than the first threshold.

[0358] Optionally, the color protection area includes a skin color area and a high saturation area, the transition area includes a near skin color area and a near high saturation area, and the non-color protection area is a chromaticity area outside the color protection area and the transition area in the color space. The determination module 1702 is used to:

[0359] In a case where the pixel color value is located in an overlapping area between the near-high saturation area and the skin color area, determining, according to the color enhancement mapping relationship, the third color enhancement coefficient corresponding to the pixel color value in the near-high saturation area and the first color enhancement coefficient corresponding to the skin color area;

[0360] Determine the minimum value of the first color enhancement coefficient and the third color enhancement coefficient, or a fusion value obtained by fusion of the first color enhancement coefficient and the third color enhancement coefficient, as the pixel color enhancement coefficient of the pixel point; or

[0361] In a case where the pixel color value is located in an overlapping area between the near high saturation area and the near skin color area, determining, according to the color enhancement mapping relationship, the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area;

[0362] The minimum value of the third color enhancement coefficient corresponding to the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area, or the fusion value of the third color enhancement coefficient corresponding to the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area, is determined as the pixel color enhancement coefficient of the pixel point.

[0363] Optionally, the color space is a YUV color space, and the color enhancement module is used to:

[0364] The product of the pixel color value corresponding to each pixel point in the image in the YUV color space and the pixel color enhancement coefficient is determined as the pixel enhancement color value corresponding to the pixel point; the pixel enhancement color value corresponding to each pixel point is converted to the RGB color space to obtain the color enhanced image; or

[0365] The initial color enhanced image corresponding to the image is determined by using a lookup table corresponding to color enhancement and the RGB values ​​of each pixel in the image; based on the pixel color values ​​corresponding to each pixel in the image frame in the YUV color space, a coefficient image composed of the pixel color enhancement coefficients corresponding to each pixel is determined, and the coefficient image has the same size as the initial color enhanced image; the coefficient image is merged with the initial color enhanced image to obtain the color enhanced image.

[0366] Optionally, the image is a video frame extracted from a video, the color-enhanced image is a color-enhanced video frame, and the device further includes a transmission module, which is used to:

[0367] Based on the color-enhanced video frame, generating a color-enhanced video;

[0368] When a color enhancement request sent by a terminal is received, the color enhanced video is transmitted to the terminal.

[0369] See also Fig.18 , Fig.18 1 is a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application. The computer device can be implemented as a server or a terminal in the above embodiments.

[0370] Specifically, the computer device 1800 includes a central processing unit (CPU) 1801, a system memory 1804 including a random access memory 1802 and a read-only memory 1803, and a system bus 1805 connecting the system memory 1804 and the central processing unit 1801. The computer device 1800 also includes a basic input / output system (I / O system) 1806 for facilitating information transmission between various components in the computer, and a large-capacity storage device 1807 for storing an operating system 1813, application programs 1814, and other program modules 1815.

[0371] The basic input / output system 1806 includes a display 1808 for displaying information and an input device 1809 such as a mouse and a keyboard for user inputting information. The display 1808 and the input device 1809 are connected to the central processing unit 1801 through an input / output controller 1810 connected to the system bus 1805. The basic input / output system 1806 may also include an input / output controller 1810 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 1810 also provides output to a display screen, a printer, or other types of output devices.

[0372] The mass storage device 1807 is connected to the central processing unit 1801 through a mass storage controller (not shown) connected to the system bus 1805. The mass storage device 1807 and its associated computer readable media provide non-volatile storage for the computer device 1800. That is, the mass storage device 1807 may include a computer readable medium (not shown) such as a hard disk or drive.

[0373] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include random access memory (RAM), read-only memory (ROM), flash memory or other solid-state storage technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will know that the computer storage medium is not limited to the above. The above-mentioned system memory 1804 and mass storage device 1807 can be collectively referred to as memory.

[0374] The memory stores one or more programs, and the one or more programs are configured to be executed by one or more central processing units 1801. The one or more programs contain computer instructions for implementing the above-mentioned methods. The central processing unit 1801 executes the one or more programs to implement the methods provided by the above-mentioned various method embodiments.

[0375] According to various embodiments of the present application, the computer device 1800 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 1800 can be connected to the network 1812 through the network interface unit 1811 connected to the system bus 1805, or the network interface unit 1811 can be used to connect to other types of networks or remote computer systems (not shown).

[0376] The memory also includes one or more programs, which are stored in the memory and include steps executed by a computer device in the method provided in the embodiment of the present application.

[0377] The embodiment of the present application also provides a computer-readable storage medium, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to implement the method described in the above embodiment. Optionally, the computer-readable storage medium may include: ROM, RAM, solid state drives (SSD, Solid State Drives) or optical disks, etc. Among them, RAM may include resistance random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).

[0378] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations of the above aspects.

[0379] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for color enhancement of an image, characterized in that: The method comprises: Obtaining a pixel color value of each pixel point in an image in a color space, wherein the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area; wherein, in a saturation enhancement scene, the color protection area is a chromaticity area outside a first ellipse in a UV plane, the non-color protection area is a chromaticity area inside a second ellipse in a UV plane, the transition area is a chromaticity area inside the first ellipse and outside the second ellipse, the first ellipse is an inscribed ellipse of the chromaticity area in the UV plane of a YUV color space, and the second ellipse is obtained by shrinking the first ellipse according to a second proportional coefficient; the second proportional coefficient is determined based on a transition factor and a second color enhancement coefficient corresponding to the color value in the non-color protection area, the second proportional coefficient is positively correlated with the transition factor, and negatively correlated with the second color enhancement coefficient, and the transition factor is used to control the regional distribution of the transition area before and after color enhancement; Determine a pixel color enhancement coefficient for each pixel point according to a color enhancement mapping relationship and the pixel color value, wherein the color enhancement mapping relationship represents a correspondence between a color value in the color space and a color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area; According to the pixel color enhancement coefficient of each pixel point, the color of each pixel point in the image is enhanced to obtain a color enhanced image.

2. The method according to claim 1, characterized in that The method further comprises: Dividing the color space into the color protection area, the transition area and the non-color protection area; Determine a first color enhancement coefficient corresponding to the color value in the color protection area, and a second color enhancement coefficient corresponding to the color value in the non-color protection area; A third color enhancement coefficient corresponding to the color value in the transition area is determined based on the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area, wherein the difference between the third color enhancement coefficient and the first color enhancement coefficient is positively correlated with the distance, and the difference between the second color enhancement coefficient and the third color enhancement coefficient is negatively correlated with the distance.

3. The method according to claim 2, characterized in that The dividing the color space into the color protection area, the transition area and the non-color protection area comprises: Determining a first shape area in a UV plane of the YUV color space based on a chromaticity range corresponding to a color value to be protected; Expand the first shape region on the UV plane according to a first proportional coefficient to obtain a second shape region; The chromaticity area in the UV plane that is located within the first shape area is determined as the color protection area; the chromaticity area in the UV plane that is located within the second shape area and outside the first shape area is determined as the transition area; and the chromaticity area in the UV plane that is located outside the second shape area is determined as the non-color protection area.

4. The method according to claim 3, characterized in that The method further comprises at least one of the following: Determining the first proportionality coefficient according to the image content type of the image, where different image content types correspond to different first proportionality coefficients; The first proportionality coefficient is determined according to the area ratio of the ROI region in the image, the first proportionality coefficient is positively correlated to the area ratio, and the pixel color value corresponding to the pixel point in the ROI region is located in the color protection area.

5. The method according to claim 2, characterized in that: The determining of a first color enhancement coefficient corresponding to the color value in the color protection area includes: The first color enhancement coefficient is determined according to the image content type of the image, and different image content types correspond to different first color enhancement coefficients.

6. The method according to claim 2, characterized in that The determining, according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area, a third color enhancement coefficient corresponding to the color value in the transition area includes: determining an interpolation coefficient based on the distance from the color value in the transition area to the color protection area; The first color enhancement coefficient and the second color enhancement coefficient are interpolated according to the interpolation coefficient to obtain the third color enhancement coefficient corresponding to the color value in the transition area.

7. The method according to claim 6, characterized in that The transition area is a chromaticity area in the UV plane of the YUV color space, which is located outside the first rectangular area and within the second rectangular area, and the interpolation coefficient is determined based on the distance from the color value in the transition area to the color protection area, including: Determine the distances from each color value on the second rectangular area to the color protection area as a distance set; Determine the minimum distance in the distance set as a distance threshold; Determine the ratio of the distance from the color value in the transition area to the color protection area to the distance threshold as an interpolation coefficient; The method further comprises: The transition area is corrected, wherein the distance from the color value in the corrected transition area to the color protection area is less than the distance threshold.

8. The method according to any one of claims 3 to 7, characterized in that: The color protection area is a skin color area, the non-color protection area is a non-skin color area, and the transition area is a near-skin color area between the skin color area and the non-skin color area.

9. The method according to claim 1, characterized in that: The method further comprises: The transition factor is determined based on the saturation of the image, and the saturation is negatively correlated to the transition factor.

10. The method according to claim 2, characterized in that The determining, according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the transition area to the color protection area, a third color enhancement coefficient corresponding to the color value in the transition area includes: A first circle is determined by taking the minor axis of the first ellipse as a radius and the point with the lowest saturation in the UV plane as a center; a second circle is determined by taking the minor axis of the second ellipse as a radius and the point with the lowest saturation in the UV plane as a center; Determine a chromaticity region in the UV plane that is within the first circle and outside the second circle as a ring zone region; Determining an affine transformation relationship between the annular zone and the transition zone; Mapping the distance from the color value in the transition area to the color protection area to the distance from the color value in the annular area to the center of the circle, wherein the color value in the transition area is mapped to the color value in the annular area based on the affine transformation relationship; The third color enhancement coefficient corresponding to the color value in the transition area is determined according to the first color enhancement coefficient, the second color enhancement coefficient, and the distance from the color value in the annular area to the center of the circle.

11. The method according to any one of claims 1 to 10, characterized in that: The color protection area is a high-saturation area with a saturation greater than a first threshold, the non-color protection area is a low-saturation area with a saturation less than a second threshold, and the transition area is a near-high-saturation area with a saturation greater than the second threshold and less than the first threshold.

12. The method according to claim 2, characterized in that: The color protection area includes a skin color area and a high saturation area, the transition area includes a near skin color area and a near high saturation area, and the non-color protection area is a chromaticity area outside the color protection area and the transition area in the color space; The step of determining the pixel color enhancement coefficient of the pixel point according to the color enhancement mapping relationship and the pixel color value includes: In a case where the pixel color value is located in an overlapping area between the near-high saturation area and the skin color area, determining, according to the color enhancement mapping relationship, the third color enhancement coefficient corresponding to the pixel color value in the near-high saturation area and the first color enhancement coefficient corresponding to the skin color area; Determine the minimum value of the first color enhancement coefficient and the third color enhancement coefficient, or a fusion value obtained by fusion of the first color enhancement coefficient and the third color enhancement coefficient, as the pixel color enhancement coefficient of the pixel point; or In a case where the pixel color value is located in an overlapping area between the near high saturation area and the near skin color area, determining, according to the color enhancement mapping relationship, the third color enhancement coefficient corresponding to the pixel color value in the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area; The minimum value of the third color enhancement coefficient corresponding to the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area, or the fusion value of the third color enhancement coefficient corresponding to the near high saturation area and the third color enhancement coefficient corresponding to the near skin color area, is determined as the pixel color enhancement coefficient of the pixel point.

13. The method according to claim 1, characterized in that The step of enhancing the color of each pixel in the image according to the pixel color enhancement coefficient of each pixel to obtain a color enhanced image includes: Determine the product of the pixel color value corresponding to each pixel point in the image in the YUV color space and the pixel color enhancement coefficient as the pixel enhancement color value corresponding to the pixel point; convert the pixel enhancement color value corresponding to each pixel point into the RGB color space to obtain the color enhanced image; or The initial color enhanced image corresponding to the image is determined by using a lookup table corresponding to color enhancement and the RGB values ​​of each pixel in the image; based on the pixel color values ​​corresponding to each pixel in the image frame in the YUV color space, a coefficient image composed of the pixel color enhancement coefficients corresponding to each pixel is determined, and the coefficient image has the same size as the initial color enhanced image; the coefficient image is merged with the initial color enhanced image to obtain the color enhanced image.

14. The method according to claim 1, characterized in that The image is a video frame extracted from a video, the color-enhanced image is a color-enhanced video frame, and the method further includes: Based on the color-enhanced video frame, generating a color-enhanced video; When a color enhancement request sent by a terminal is received, the color enhanced video is transmitted to the terminal.

15. A device for enhancing the color of an image, characterized in that: The device comprises: An acquisition module is used to acquire the pixel color value of each pixel in the image in a color space, wherein the color space includes a color protection area, a non-color protection area, and a transition area between the color protection area and the non-color protection area; wherein, in a saturation enhancement scene, the color protection area is a chromaticity area outside a first ellipse in a UV plane, the non-color protection area is a chromaticity area inside a second ellipse in a UV plane, the transition area is a chromaticity area inside the first ellipse and outside the second ellipse, the first ellipse is an inscribed ellipse of the chromaticity area in the UV plane of a YUV color space, and the second ellipse is obtained by shrinking the first ellipse according to a second proportional coefficient; the second proportional coefficient is determined based on a transition factor and a second color enhancement coefficient corresponding to the color value in the non-color protection area, the second proportional coefficient is positively correlated with the transition factor, and negatively correlated with the second color enhancement coefficient, and the transition factor is used to control the regional distribution of the transition area before and after color enhancement; A determination module, configured to determine a pixel color enhancement coefficient of each pixel point according to a color enhancement mapping relationship and the pixel color value, wherein the color enhancement mapping relationship represents a correspondence between a color value in the color space and a color enhancement coefficient, and the color enhancement coefficient corresponding to the color value in the transition area is between the color enhancement coefficient corresponding to the color value in the color protection area and the color enhancement coefficient corresponding to the color value in the non-color protection area; The color enhancement module is used to enhance the color of each pixel in the image according to the pixel color enhancement coefficient of each pixel to obtain a color enhanced image.

16. A computer device, characterized in that: The computer device comprises: a processor and a memory, wherein at least one computer instruction is stored in the memory, and the at least one computer instruction is loaded and executed by the processor to implement the color enhancement method of an image as claimed in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer instruction, and the computer instruction is loaded and executed by a processor to implement the color enhancement method of an image as described in any one of claims 1 to 14.

18. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the color enhancement method of an image as described in any one of claims 1 to 14.

Images