A method, system and device for real-time video image processing of human skin color recognition

Abstract

The application discloses a portrait skin color recognition method suitable for real-time video image processing, which comprises the following steps: calculating the skin color proportion η of each image pixel of a frame of image data; converting the RGB data of the current image pixel data into XYZ data; converting the XYZ data of the current image pixel data into LAB data; converting the LAB data of the current image pixel data into LCh data; determining the first landscape proportion η of the pixel based on the LCH data of the current image pixel data; determining the maximum chroma C of the pixel based on the LCH data of the current image pixel data; determining the second landscape proportion η of the pixel based on the chroma C in the LCH data of the current image pixel data and the maximum chroma C; obtaining the final landscape proportion η of the current pixel based on the first landscape proportion η and the second landscape proportion η; and the skin color proportion η is 1-η. skin h * max * * max c h c scene skin scene ​​​​​​​​​​​

Description

Technical Field

[0001] This invention relates to the field of video image processing technology, and in particular to a method, system, and apparatus for human skin color recognition suitable for real-time video image processing. Background Technology

[0002] In the field of video image processing, skin color recognition is often used to distinguish between portraits and landscapes, and then the portrait and landscape components are processed separately. For example, in saturation enhancement techniques, it is usually necessary to keep the saturation of the portrait's skin unchanged while enhancing the saturation of only the landscape portion, thus making the image vivid and colorful while avoiding image quality distortion. Therefore, how to identify the skin color portion of an image has practical significance.

[0003] Traditional facial skin color recognition uses a "feature vector matching" method: after image preprocessing, a set of feature vectors related to skin color is extracted, and then the skin color region is determined based on the distance between this feature vector and a pre-trained set of skin color feature vectors. This method is accurate, but computationally complex, and requires knowledge of the entire image area before skin color determination. However, in video processing scenarios, pixels are input to the image processing unit line by line. This method cannot determine the skin color region before receiving the entire image, so the "feature vector matching" method needs to cache a complete frame. This limitation not only results in extremely high hardware storage space but also introduces at least one frame of processing latency. Therefore, this method is often suitable for non-real-time image processing scenarios.

[0004] Another simplified skin color recognition method uses a lookup table: first, the image is converted from RGB space to YCbCr space, and then (Cb, Cr) is used as the index of the lookup table to identify the proportion of the scene. This simplified method relies only on the RGB data of the current pixel for skin color recognition, so it has good real-time performance. However, since the skin color distribution in YCbCr space is not continuous, this method requires a very precise quantized index (Cb, Cr), resulting in a large number of indices and thus significant hardware overhead.

[0005] Therefore, it is evident that there is a need for a new facial recognition method suitable for real-time video image processing in the existing technology, in order to achieve more effective facial skin recognition. Summary of the Invention

[0006] The technical objective of this invention is to provide a skin tone recognition method for images suitable for real-time video processing. This method can quickly calculate the proportion of skin tone contained in each pixel of an image. Compared with existing technologies, this invention calculates the proportion of skin tone in a pixel based solely on the RGB data of the current pixel, thus offering better real-time performance and making it suitable for real-time video processing. Furthermore, this invention eliminates the need for large lookup tables, resulting in lower storage resource consumption.

[0007] Based on the above technical objectives, the present invention provides a method for facial skin color recognition suitable for real-time video image processing, the method comprising:

[0008] S100, calculate the skin tone ratio η for each pixel of a frame of image data. skin Calculation;

[0009] S101, convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel;

[0010] S102, convert the XYZ data of the current image pixel data from the XYZ color system to the LAB color system to generate the LAB data of the current image pixel;

[0011] S103, convert the LAB data of the current image pixel data from the LAB color system to the LCH color space to generate the LCH data of the current image pixel;

[0012] S104, determine the first scene ratio η of the pixel based on the LCH data of the current image pixel data. h ;

[0013] S105, determine the maximum chromaticity C of the pixel based on the LCH data of the current image pixel data. * max ;

[0014] S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c ;

[0015] S107, based on the first scene scale η h Second scenery ratio η c Get the final scene scale η of the current pixel scene The skin color ratio η skin For 1-η scene .

[0016] In one embodiment, the conversion formula used to convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel is as follows:

[0017] Where R is the R color data component in the RGB color system, G is the G color data component in the RGB color system, B is the B color data component in the RGB color system; X is the X data component in the XYZ color system, Y is the Y data component in the XYZ color system, and Z is the Z data component in the XYZ color system.

[0018] In one embodiment, the formula used to convert XYZ data into LAB data is:

[0019] ;

[0020] in, For the reference white point's XYZ coordinates, under CIE D65 illumination conditions: , The function is as follows:

[0021] ;

[0022] L * For the L* data component in the LAB color system, a * For the LAB color system, a * Data components, b * b under the LAB color system * Data components.

[0023] In one embodiment, the formula used to convert LAB data to LCh data is:

[0024] ;

[0025] in, In the LCh color space Data components, In the LCh color space Data components, Hue angle in the LCh color space Data components.

[0026] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0027] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0028] Figure 1This is a schematic diagram of the human skin color recognition method of the present invention;

[0029] Figure 2 This is a schematic diagram of the hue angle division of the present invention;

[0030] Figure 3 This is a schematic diagram illustrating the relationship between the hue angle and the first depth of field ratio of the present invention;

[0031] Figure 4 This is a schematic diagram illustrating the relationship between the hue angle and maximum chromaticity of the present invention;

[0032] Figure 5 This is a schematic diagram illustrating the relationship between chromaticity and the second depth of field in this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0034] It should be understood that when an element or layer is referred to as "on," "adjacent to," "connected to," or "coupled to" other elements or layers, it may be directly on, adjacent to, connected to, or coupled to other elements or layers, or there may be intervening elements or layers. Conversely, when an element is referred to as "directly on," "directly adjacent to," "directly connected to," or "directly coupled to" other elements or layers, there are no intervening elements or layers. It should be understood that although the terms first, second, third, etc., may be used to describe various elements, components, areas, layers, and / or portions, these elements, components, areas, layers, and / or portions should not be limited by these terms. These terms are only used to distinguish one element, component, area, layer, or portion from another element, component, area, layer, or portion. Therefore, without departing from the teachings of this invention, the first element, component, area, layer, or portion discussed below may be referred to as a second element, component, area, layer, or portion. And when a second element, component, area, layer, or portion is discussed, it does not imply that the first element, component, area, layer, or portion necessarily exists in this invention.

[0035] Spatial relation terms such as “below,” “under,” “below,” “under,” “above,” “above,” etc., are used herein for convenience of description to describe the relationship between one element or feature shown in the figure and other elements or features. It should be understood that, in addition to the orientation shown in the figure, spatial relation terms are intended to also include different orientations of the device in use and operation. For example, if the device in the figure is flipped, then the element or feature described as “below,” “under,” or “below” other elements or features will be oriented “above” other elements or features. Therefore, the exemplary terms “below” and “under” can include both above and below orientations. The device may be otherwise oriented (rotated 90 degrees or otherwise) and the spatial descriptive terms used herein will be interpreted accordingly.

[0036] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and / or “including,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0037] Example 1

[0038] like Figure 1 As shown, the human skin color recognition method for real-time video image processing in this embodiment includes:

[0039] S100, calculate the skin tone ratio η for each pixel of a frame of image data. skin Calculation;

[0040] S101, the RGB data of the current image pixel data is converted from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel. Specifically, after receiving the RGB data of a certain image pixel, the image processing module first performs a color space transformation, converting it to the XYZ color system. For example, assuming RGB is 8-bit data, the conversion formula is:

[0041] ;

[0042] S102, convert the XYZ color system of the current image pixel data to the LAB color system to generate the LAB data of the current image pixels. Specifically, the XYZ space data is converted to CIELAB space data using the following formula:

[0043] ;

[0044] in, For the reference white point's XYZ coordinates, under CIE D65 illumination conditions: , The function is as follows:

[0045] ;

[0046] S103, convert the LAB data of the current image pixel data from the LAB color system to the LCH color space to generate LCh data of the current image pixel. Specifically, the LAB data is converted to LCh data using the following formula:

[0047] ;

[0048] S104, determine the first scene ratio η of the pixel based on the LCH data of the current image pixel data. h Specifically, Figure 2 As shown, when the hue angle h satisfies or At that time, it was considered to be entirely skin color, that is, the proportion of the scenery. =0; when the hue angle h satisfies Within the range, it is considered to be entirely scenery, i.e., the proportion of scenery. =1; when the hue angle h satisfies or When within the range, it is considered to be in the transition zone between skin color and scenery, according to the hue angle h. Figure 3 The broken line shown is used to calculate the corresponding scenery scale through interpolation. Obviously, within this interval .

[0049] S105, determine the maximum chromaticity C of the pixel based on the LCH data of the current image pixel data. * max Specifically, the maximum chromaticity of a pixel in the CIE space is calculated based on the hue angle h in the LCh data, at that hue angle. Here h and The relationship can be approximated as Figure 4 The defined polyline consists of 9 connected vertices, in the format of... The corresponding coordinates are respectively , , , , , , , , .

[0050] S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c Specifically, based on the chromaticity in the LCh data. and the maximum chromaticity obtained in step S105 Calculate the scene ratio of pixels , specifically Figure 5 As shown: When chromaticity Less than At that time, it was considered to be entirely skin color, that is, the proportion of the scenery. =0; when chromaticity Greater than At that time, it was considered to be entirely scenery, that is, the proportion of scenery. =1; when chromaticity exist Within the range, it is considered to be the transition zone between skin color and scenery, according to Figure 5 The broken line shown is used to calculate the corresponding scenery scale through interpolation. Obviously, within this interval ;

[0051] S107, based on the first scene scale η h Second scenery ratio η c Get the final scene scale η of the current pixel scene Specifically, based on the proportions of the first scene. Second scenery ratio Calculate the final scenery ratio for ,and No more than 1, that is At this point, the skin tone ratio in that pixel... for .

[0052] This invention can be any possible system, method, and / or computer program product at the level of integrated technical detail. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0053] A computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable optical disc read-only memory (CD-ROM), a digital universal disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or raised structure in a notch on which instructions are recorded, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0054] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. This network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0055] Computer-readable program instructions used to perform the operations of this invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in one or more programming languages ​​and any combination of procedural programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet through an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute the computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry and thereby perform aspects of the invention.

[0056] This document describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0057] These computer-readable program instructions can be provided to a computer's processor or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, create a mechanism for implementing flowcharts and / or blocks. Figure 1 Means for the functions / actions specified in one or more boxes. These computer-readable program instructions may also be stored in a computer-readable storage medium capable of guiding a computer, a programmable data processing apparatus and / or other apparatus operating in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture comprising instructions for implementing aspects of the functions / actions specified in the flowchart and / or block diagram boxes.

[0058] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other apparatus to cause a series of operational steps to be performed on the computer, other programmable devices or other apparatus for producing computer-implemented processes, such that the instructions executed on the computer, other programmable devices or other apparatus perform the functions / actions specified in the flowchart and / or block diagram boxes.

[0059] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in a block may occur outside the order indicated in the diagram. For example, two blocks shown consecutively may actually be completed as a single step, executed concurrently, substantially concurrently, in a manner that overlaps partially or entirely in time, depending on the functions involved, or sometimes these blocks may be executed in reverse order. It will also be noted that each block illustrated in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0060] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A method for facial skin color recognition suitable for real-time video image processing, characterized in that, The method includes: S100, calculate the skin tone ratio η for each pixel of a frame of image data. skin Calculation; S101, convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel; S102, convert the XYZ data of the current image pixel data from the XYZ color system to the LAB color system to generate the LAB data of the current image pixel; S103, convert the LAB data of the current image pixel data from the LAB color system to the LCH color space to generate the LCH data of the current image pixel; S104, determine the first scene ratio η of the pixel based on the LCH data of the current image pixel data. h ; S105, determine the maximum chromaticity C of the pixel based on the LCH data of the current image pixel data. * max ; S106, Chromaticity C in LCH data based on current image pixel data * and maximum chromaticity C * max Determine the second color ratio η of the pixel c ; S107, based on the first scene scale η h Second scenery ratio η c Calculate the final scene scale for ,and Not exceeding 1; the skin color ratio η skin For 1-η scene ; The conversion formula used to convert the RGB data of the current image pixel data from the RGB color system to the XYZ color system to generate the XYZ data of the current image pixel is as follows: ； Where R represents the R color data component in the RGB color system, G represents the G color data component in the RGB color system, and B represents the B color data component in the RGB color system; X represents the X data component in the XYZ color system, Y represents the Y data component in the XYZ color system, and Z represents the Z data component in the XYZ color system; the formula used to convert XYZ data to LAB data is: ； in, For the reference white point's XYZ coordinates, under CIE D65 illumination conditions: , The function is as follows: ； L * For the L* data component in the LAB color system, a * For the LAB color system, a * Data components, b * b under the LAB color system * Data components; the formula used to convert LAB data to LCh data is: ； in, In the LCh color space Data components, In the LCh color space Data components, Hue angle in the LCh color space Data components.

2. A computer-readable storage medium having computer instructions stored thereon, wherein, When the computer instructions are executed by the processor, they implement the steps of the method of claim 1.

Images