Portrait skin color recognition method, system and device suitable for real-time video image processing

Abstract

The invention discloses a portrait skin color recognition method suitable for real-time video image processing. The portrait skin color recognition method comprises the steps that skin color proportion eta skin calculation is carried out on each image pixel of a frame of image data; converting RGB data of the pixel data of the current image into XYZ data; converting XYZ data of the current image pixel data into LAB data; converting the LAB data of the current image pixel data into LCh data; determining a first scene ratio eta h of the pixel based on the LCH data of the current image pixel data; determining the maximum chromaticity C * max of the pixel based on the LCH data of the current image pixel data; determining a second scene ratio [eta] c of the pixel based on the chromaticity C * and the maximum chromaticity C * max in the LCH data of the current image pixel data; obtaining a final scene proportion eta scene of the current pixel based on the first scene proportion eta h and the second scene proportion eta c; the skin color proportion eta skin is 1-eta scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image processing, and in particular to a portrait skin color recognition method, system and device suitable for real-time video image processing. BACKGROUND

[0002] In the field of video image processing, skin color recognition is often used to distinguish between portraits and landscapes, and then the portrait component and the landscape component are processed respectively. For example, in the saturation enhancement technology, it is usually necessary to keep the saturation of the skin of the portrait unchanged and only enhance the saturation of the landscape part, so as to make the image lively and bright and avoid image distortion. Therefore, how to recognize the skin color part of the picture has practical significance.

[0003] The traditional portrait skin color recognition is realized by the "feature vector matching" method: after pre-processing the image, the feature vector set related to the skin color is extracted, and then the skin color region is determined according to the distance between the feature vector and the pre-trained skin color feature vector set. This method is accurate in determination, but has high computational complexity, and the image of the entire region needs to be known before skin color determination. However, in the video processing scene, pixels are input to the image processing unit in a line-by-line scanning manner. This method cannot determine the skin color region before receiving the entire image, so the "feature vector matching" method needs to cache a complete frame of image. This limitation not only brings extremely high hardware storage space, but also introduces at least one frame of processing delay. Therefore, this method is often suitable for non-real-time image processing scenes.

[0004] Another simple skin color recognition method is realized by a lookup table: first, the image is converted from the RGB space to the YCbCr space, and then the (Cb, Cr) is used as the lookup table index to identify the landscape proportion. This simple method only relies on the current pixel RGB data when performing skin color recognition, so it has good real-time performance, but since the skin color distribution in the YCbCr space is not continuous, this method requires very accurate quantization of the index (Cb, Cr), so it contains a large number of index numbers, which also brings a large hardware overhead.

[0005] It can be seen that there is a need for a new portrait recognition method suitable for real-time video image processing in the prior art to achieve more effective portrait skin recognition. SUMMARY

[0006] The technical purpose of the present application is to provide a picture skin color recognition method suitable for real-time video processing, which can quickly calculate the skin color proportion contained in each pixel point in the picture. Compared with the prior art, on the one hand, the present application only relies on the current pixel RGB data when calculating the skin color proportion in the pixel, so it has good real-time performance and is suitable for real-time video processing field. On the other hand, the present application does not need a large-size lookup table, and consumes less storage resources.

[0007] Based on the above technical objectives, the present invention provides a method for facial skin color recognition suitable for real-time video image processing, the method comprising:

[0008] S100, calculate the skin tone ratio η for each pixel of a frame of image data. skin Calculation;

[0009] S101, convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel;

[0010] S102, convert the XYZ data of the current image pixel data from the XYZ color system to the LAB color system to generate the LAB data of the current image pixel;

[0011] S103, convert the LAB data of the current image pixel data from the LAB color system to the LCH color space to generate the LCH data of the current image pixel;

[0012] S104, determine the first scene ratio η of the pixel based on the LCH data of the current image pixel data. h ;

[0013] S105, determine the maximum chromaticity C of the pixel based on the LCH data of the current image pixel data. * max ;

[0014] S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c ;

[0015] S107, based on the first scene scale η h Second scenery ratio η c Get the final scene scale η of the current pixel scene The skin color ratio η skin For 1-η scene .

[0016] In one embodiment, the conversion formula used to convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel is as follows:

[0017] ; wherein R is the R color data component under the RGB colorimetric system, G is the G color data component under the RGB colorimetric system, B is the B color data component under the RGB colorimetric system; X is the X data component under the XYZ colorimetric system, Y is the Y data component under the XYZ colorimetric system, Z is the Z data component under the XYZ colorimetric system.

[0018] In one embodiment, the formula for converting XYZ data to LAB data is:

[0019] ;

[0020] wherein, X Y Z is the XYZ coordinate of the reference white point, which is , is the following function:

[0021] ;

[0022] L * is the L* data component under the LAB colorimetric system, a * is the a * data component under the LAB colorimetric system, b * is the b * data component under the LAB colorimetric system.

[0023] In one embodiment, the formula for converting LAB data to LCh data is:

[0024] ;

[0025] wherein, is the data component in the LCh color space, is the data component in the LCh color space, is the hue angle data component in the LCh color space.

[0026] Additional features and advantages of the application will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the application. In the drawings:

[0028] Figure 1is a flowchart of a human skin color recognition method according to the present application;

[0029] Figure 2 is a color angle division diagram according to the present application;

[0030] Figure 3 is a color angle and first depth of field ratio relationship diagram according to the present application;

[0031] Figure 4 is a color angle and maximum chroma relationship diagram according to the present application;

[0032] Figure 5 is a chroma and second depth of field ratio relationship diagram according to the present application. DETAILED DESCRIPTION

[0033] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings.

[0034] It should be understood that when an element or layer is referred to as being "on", "adjacent", "connected to", or "coupled to" another element or layer, it can be directly on, adjacent, connected or coupled to the other element or layer, or intervening elements or layers can be present. In contrast, when an element is referred to as being "directly on", "directly adjacent", "directly connected to", or "directly coupled to" another element or layer, then there are no intervening elements or layers present. It will be understood that, although the terms first, second, third, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the present application, and, similarly, a second element, component, region, layer or section discussed below could be termed a first element, component, region, layer or section without departing from the teachings of the present application.

[0035] Spatially relative terms, such as "under", "below", "lower", "on", "above", "upper", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is inverted, then a dependent element or feature it is described as "below" or "beneath" another element or feature is oriented upward from the latter. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0036] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0037] Example 1

[0038] As Figure 1 shown, the human skin color recognition method suitable for real-time video image processing of the present embodiment includes:

[0039] S100, calculating the skin color ratio η of each image pixel of a frame of image data; skin

[0040] S101, converting the RGB data of the current image pixel data from the RGB colorimetric system to the XYZ colorimetric system to generate the XYZ data of the current image pixel. Specifically, after receiving the RGB data of a certain image pixel, the image processing module first performs color space conversion to change it into the XYZ colorimetric system. For example, assuming that the RGB is 8-bit data, the conversion formula is:

[0041]

[0042] S102, converting the XYZ data of the current image pixel data from the XYZ colorimetric system to the LAB colorimetric system to generate the LAB data of the current image pixel. Specifically, the XYZ space data is converted to CIELAB space data by the following formula, which is as follows:​​

[0043] ;

[0044] wherein, are XYZ coordinates of the reference white point, under CIE D65 illumination conditions , are functions of:

[0045] ;

[0046] S103, converting the LAB data of the current image pixel data from the LAB colorimetric system to the LCH color space to generate LCh data of the current image pixel. Specifically, the LAB data is converted to LCh data by the following formula, specifically as follows:

[0047] ;

[0048] S104, determining the first scenery ratio η of the pixel based on the LCH data of the current image pixel data h . Specifically, Figure 2 as shown, when the hue angle h satisfies or , it is considered to be completely skin color, i.e. the scenery ratio is 0; when the hue angle h satisfies interval, it is considered to be completely scenery, i.e. the scenery ratio is 1; when the hue angle h satisfies or interval, it is considered to be in the transition interval of skin color and scenery, according to the broken line as shown in Figure 3 , the corresponding scenery ratio is calculated by interpolation, obviously in this interval.

[0049] S105, determining the maximum chroma C of the pixel based on the LCH data of the current image pixel data * max . Specifically, according to the hue angle h in the LCh data, the maximum chroma C of the pixel in the CIE space under the hue angle is calculated . Here, the relationship between h and can be approximated as Figure 4 defined broken line, which is connected by 9 vertices, and the format is , and the corresponding coordinates are , , , , , , , , .

[0050] S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c Specifically, based on the chromaticity in the LCh data. and the maximum chromaticity obtained in step S105 Calculate the scene ratio of pixels , specifically Figure 5 As shown: When chromaticity Less than At that time, it was considered to be entirely skin color, that is, the proportion of the scenery. =0; when chromaticity Greater than At that time, it was considered to be entirely scenery, that is, the proportion of scenery. =1; when chromaticity exist Within the range, it is considered to be the transition zone between skin color and scenery, according to Figure 5 The broken line shown is used to calculate the corresponding scenery scale through interpolation. Obviously, within this interval ;

[0051] S107, based on the first scene scale η h Second scenery ratio η c Get the final scene scale η of the current pixel scene Specifically, based on the proportions of the first scene. Second scenery ratio Calculate the final scenery ratio for ,and No more than 1, that is At this point, the skin tone ratio in that pixel... for .

[0052] This invention can be any possible system, method, and / or computer program product at the level of integrated technical detail. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0053] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanism that reads a set of instructions from a set of instructions, such as a groove in a disc, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0054] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions into the computing / processing device for storage in a computer readable storage medium within the respective computing / processing device.

[0055] Computer readable program instructions for carrying out operations of the present application can be in assembly code, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state set data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the "C" programming language. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0056] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0057] These computer readable program instructions can be provided to a processor of a computer, or other programmable data processing apparatus, to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Figure 1 These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored

[0058] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0059] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks.

[0060] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks.

[0061] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks.

[0062] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks.

Claims

1. A method for facial skin color recognition suitable for real-time video image processing, characterized in that, The method includes: S100, calculate the skin tone ratio η for each pixel of a frame of image data. skin Calculation; S101, convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel; S102, convert the XYZ data of the current image pixel data from the XYZ color system to the LAB color system to generate the LAB data of the current image pixel; S103, convert the LAB data of the current image pixel data from the LAB color system to the LCH color space to generate the LCH data of the current image pixel; S104, determine the first scene ratio η of the pixel based on the LCH data of the current image pixel data. h ; S105, determine the maximum chromaticity C of the pixel based on the LCH data of the current image pixel data. * max ; S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c ; S107, based on the first scene scale η h Second scenery ratio η c Get the final scene scale η of the current pixel scene The skin color ratio η skin For 1-η scene .

2. The facial skin color recognition method according to claim 1, characterized in that, The conversion formula used to convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel is as follows: ； Where R is the R color data component in the RGB color system, G is the G color data component in the RGB color system, B is the B color data component in the RGB color system; X is the X data component in the XYZ color system, Y is the Y data component in the XYZ color system, and Z is the Z data component in the XYZ color system.

3. The facial skin color recognition method according to claim 1, characterized in that, The formula used to convert XYZ data to LAB data is: ； in, For the reference white point's XYZ coordinates, under CIE D65 illumination conditions: , The function is as follows: ； L * For the L* data component in the LAB color system, a * For the LAB color system, a * Data components, b * b under the LAB color system * Data components.

4. The facial skin color recognition method according to claim 1, characterized in that, The formula used to convert LAB data to LCh data is: ； in, In the LCh color space Data components, In the LCh color space Data components, Hue angle in the LCh color space Data components.

5. A computer-readable storage medium having computer instructions stored thereon, wherein, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-4.

Images