Method, device and system for processing face image

Abstract

The application provides a face image processing method, device and system. The method comprises the following steps: obtaining an original image; selecting a first region of the original image; dividing the original image according to skin color information of the first region; and obtaining a corresponding target face image based on the skin color information. The method not only realizes clearer display of portrait details in various scenes, but also shows the stereoscopic effect of deep skin color portraits, and achieves consistency of portrait shooting in different environments, thereby improving the dynamic range of the face in portrait shooting, especially in deep skin color portrait shooting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a face image processing method, device and system. BACKGROUND

[0002] HDR (High-Dynamic Range, high dynamic range image) is a technology used to make the camera see the characteristics of the image with the development of image processing technology. Compared with ordinary images, it can provide more dynamic range and image details, according to different exposure time LDR (Low-Dynamic Range, low dynamic range image), and use the LDR image corresponding to the best details of each exposure time to synthesize the final HDR image. It can better reflect the visual effect in the real environment.

[0003] However, most of the current HDR methods are for the HDR of the whole picture scene, and rarely combined with portrait detection.

[0004] Even some HDR methods related to single portrait detection only process the portrait multiple times to synthesize the clearest pixel points. However, it is difficult to quantitatively analyze how the pixel points are the clearest, and even if all the pixel points of the portrait part are clear, it cannot guarantee that the synthesized face part is problem-free, for example, the stereoscopic effect of the portrait may disappear, the dynamic range of the portrait may decrease, and the portrait may be flat. SUMMARY

[0005] The present application provides a face image processing method, device and system to achieve clearer display of portrait details in various scenes, and can also show the stereoscopic effect of dark-skinned portraits, and achieve consistency of portrait shooting in different environments, thereby improving the dynamic range of the face during portrait shooting, especially dark-skinned portrait shooting.

[0006] In a first aspect, the present application provides a face image processing method, comprising:

[0007] obtaining an original image;

[0008] selecting a first region of the original image;

[0009] dividing the original image according to skin color information according to the first region;

[0010] obtaining a corresponding target face image based on the skin color information.

[0011] In a possible design, before selecting the first region of the original image, the method further comprises:

[0012] Selecting a face region from the original image, if the brightness of the face region is not less than a preset target value, the corresponding original image is taken as a standard image, and the current exposure T is stored.

[0013] In a possible design, a first region of the original image is selected, including:

[0014] Features of the standard image are extracted, the standard image is divided into a plurality of sub-regions, and the first region is selected from the sub-regions, where the first region includes a forehead region corresponding to a face.

[0015] In a possible design, the method further includes:

[0016] According to the current exposure T, each of the sub-regions is subjected to positive exposure and negative exposure processing to obtain a plurality of alternative sub-regions.

[0017] In a possible design, the division of skin color information of the original image according to the first region includes:

[0018] Features of the first region are matched with features of the alternative sub-regions to obtain at least one corresponding alternative sub-region as a matching sub-region;

[0019] The skin color category corresponding to the first region is determined according to the matching sub-region.

[0020] In a possible design, the corresponding target face image is obtained based on the skin color information, including:

[0021] The matching sub-region corresponding to each sub-region in the face region is obtained according to the skin color category;

[0022] All the matching sub-regions are fused to obtain the target face image.

[0023] In a possible design, before the skin color category corresponding to the first region is determined according to the matching sub-region, the method further includes:

[0024] A one-to-one correspondence between the matching sub-region and the skin color category corresponding to the first region is obtained through training of a training data set.

[0025] In a possible design, after the corresponding target face image is obtained based on the skin color information, the method further includes:

[0026] The target face image is subjected to smoothing processing and is displayed on a display interface.

[0027] In a possible design, the original image is obtained, including:

[0028] Face detection is performed in the preview interface. If no face region is detected, a photographing mode is entered, and photographing light adjustment is performed according to a preset light measurement mode to obtain an original image.

[0029] In a second aspect, an apparatus for face image processing is provided, and the apparatus comprises:

[0030] An obtaining module is configured to obtain an original image.

[0031] An input module is configured to input the original image as an input of a skin color face model. The skin color face model refers to: performing skin color classification on the original image according to a forehead region of a face, and obtaining a target face image based on the skin color classification.

[0032] An output module is configured to output the target face image through the skin color face model.

[0033] In a third aspect, a system for face image processing is provided, and the system comprises a memory and a processor. The memory stores executable instructions of the processor. The processor is configured to execute the executable instructions to perform the method for face image processing according to any one of the first aspect.

[0034] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the method for face image processing according to any one of the first aspect is implemented.

[0035] The present application provides a method, apparatus and system for face image processing. The method comprises: obtaining an original image; selecting a first region of the original image; performing skin color information classification on the original image according to the first region; and obtaining a target face image based on the skin color information. The method not only realizes clearer display of portrait details in various scenes, but also performs three-dimensional expression of deep skin color portraits, and achieves consistency of portrait photographing in different environments, thereby improving the dynamic range of the face in portrait photographing, especially deep skin color portrait photographing. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0037] Figure 1 An application scenario diagram of the present application;

[0038] Figure 2 A flow chart of the method for processing a face image provided for the first embodiment of the present application is shown in FIG. 1.

[0039] Figure 3 A structural schematic diagram of a face model provided for the second embodiment of the present application is shown in FIG. 2.

[0040] Figure 4 A structural schematic diagram of the device for processing a face image provided for the third embodiment of the present application is shown in FIG. 3.

[0041] Figure 5 A structural schematic diagram of the device for processing a face image provided for the fourth embodiment of the present application is shown in FIG. 4.

[0042] Figure 6 A structural schematic diagram of the system for processing a face image provided for the fifth embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0043] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0044] The terms "first", "second", "third", "fourth" and the like (if any) in the description, claims and above drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0045] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below in connection with the drawings.

[0046] HDR(High Dynamic Range, high dynamic range) is a kind of image post-processing technology, which is a kind of image mapping technology beyond the brightness range that the display can represent. Mapping a wide brightness range to the brightness range that paper or screen can represent is similar to the camera exposure function, and the human eye also has a similar function. By the aperture of the camera, the amount of light entering the light sensor can be controlled, and the light and shade of the light sensor after certain processing can obtain a convincing photo.

[0047] However, most of the existing light measurement modes are HDR of the whole picture scene, and few of them are combined with portrait detection. Even some portrait HDRs involving individual portrait detection are only multiple processing of the portrait, and the most clear pixel points are synthesized.

[0048] Figure 1 An application scenario diagram of the present application is shown in FIG. 1. Currently, users use smart devices such as mobile phones, tablet computers, and iPads to take pictures of scenes to obtain images with appropriate exposure, as shown in FIG. 1. Figure 1 As shown in FIG. 1, a user uses a mobile phone 11 to take pictures of a scene including a portrait. Due to the different exposure of each scene, the received light is different, resulting in a picture that is bright and dark, especially the shooting of deep skin color portraits cannot meet the requirements of high dynamic range of portraits. Therefore, the light measurement mode of the present application is needed for scene shooting, especially for obtaining deep skin color portraits, and then outputting a target face image, i.e., a high dynamic face image 12, according to the skin color face model, thereby improving the shooting experience of deep skin color groups.

[0049] Figure 2 A flowchart of the method for processing a face image provided by the first embodiment of the present application is shown in FIG. 2. Figure 2 As shown in FIG. 2, the method for processing a face image in the present embodiment can include:

[0050] S101, obtaining an original image.

[0051] Specifically, when taking a picture through a terminal device such as a mobile phone, face detection is performed on the preview interface (for example, the mobile phone screen). If no face is detected, the shooting mode is entered, and the shooting light is adjusted according to the preset light measurement mode to obtain an original image. The original image includes face information.

[0052] The light measurement mode refers to the way of testing the reflectivity of the camera and the subject, measuring the brightness of light, and is the process of realizing correct exposure of the camera. The advantage of through-the-lens light measurement is that the size of the light directly reflected by the scene is seen. There are mainly four light measurement modes, namely, average light measurement, local light measurement, point light measurement, and central key average light measurement. Generally, the consistency of the overall exposure of the original image is poor, and the result is that the image is bright and dark.

[0053] S102, selecting a first region of the original image.

[0054] Specifically, in an alternative embodiment, before selecting the first region of the original image, further comprising: selecting a face region from the original image, if the brightness of the detected face region is not less than a preset target value, then the corresponding original image is taken as a standard image, and the current exposure T is stored.

[0055] For example, based on the prior art method, the face region is selected from the original image, and then the brightness of the face region is detected. If the brightness is not less than a preset target value AE, the original image is taken as a standard image, for example, reference Figure 1 The picture displayed in the mobile phone 11, and store the original image corresponding to the current exposure T, the current exposure indicates the exposure degree of the image, that is, how much light is accepted. The more light is accepted, that is, the higher the exposure, the brighter the image; the lower the exposure, the darker the image. In this embodiment, the preset target value is not limited.

[0056] Further, the first region of the original image is obtained, including: extracting the features of the standard image, dividing the standard image into a plurality of sub-regions, and selecting the first region from the sub-regions, wherein the first region includes the forehead region corresponding to the face. Further comprising: according to the current exposure T, each sub-region is subjected to positive exposure and negative exposure processing to obtain a plurality of alternative sub-regions.

[0057] Specifically, the features of the original image corresponding to the standard image are extracted, and the standard image is divided into a plurality of sub-regions, for example, the standard image is divided into a plurality of sub-regions, and the face region is divided into M sub-regions, wherein the sub-region includes the first region, for example, the forehead region of the face. Further, a plurality of frames of exposure are performed based on the current exposure T, and each sub-region of the standard image is subjected to a small STEP of positive exposure and negative exposure processing, for example, each of the positive exposure and the negative exposure is processed X times, and 2X+1 corresponding alternative sub-regions are obtained. For example, after X times of positive exposure and negative exposure processing based on the current exposure T for a sub-region, 2X+1 alternative sub-regions corresponding to the sub-region are obtained. Especially for the exposure processing of the face region, for example, after X times of positive exposure and negative exposure processing based on the current exposure T for the forehead region of the face (i.e. the first region), 2X+1 alternative sub-regions corresponding to the forehead region are obtained. In order to more comprehensively collect the brightness values of different sub-regions of the face region, the STEP of the multiple frames of exposure needs to be small enough, that is, the number of frames needs to be large enough. And cache 2X+1 alternative sub-regions collected from each sub-region, and obtain M*(2X+1) alternative sub-images from M sub-regions.

[0058] S103, dividing the original image according to the first region to obtain skin color information, matching the features of the first region with the features of the candidate sub-regions to obtain at least one corresponding candidate sub-region as a matching sub-region; and determining the skin color category corresponding to the first region according to the matching sub-region.

[0059] In this embodiment, the features of the forehead region are matched with the corresponding candidate sub-regions to obtain at least one candidate sub-region, and then the matching sub-region is obtained from the candidate sub-regions, and the skin color category, such as brown black, is determined according to the matching sub-region. Then, the matching sub-region corresponding to the other (M-1) sub-regions in the face region is obtained according to the skin color category of brown black, i.e., the matching sub-region is obtained from the 2X+1 candidate sub-regions corresponding to each sub-region, and then all the matching sub-regions are fused to obtain the target face image.

[0060] S104, obtaining the matching sub-region corresponding to each sub-region in the face region according to the skin color category; and fusing all the matching sub-regions to obtain the target face image.

[0061] Specifically, the features of the forehead region are matched with the features of the corresponding candidate sub-regions, and one or more matching candidate sub-regions can be obtained, and one of the candidate sub-regions is taken as a matching sub-region. The skin color category corresponding to the forehead region (i.e., the first region) of the face is determined according to the matching sub-region, and the matching sub-region corresponding to each sub-region in the face region is obtained according to the skin color category, and then all the matching sub-regions are fused to obtain the target face image. For example, the sub-region can include the forehead region divided by the position from the eyebrow to the hairline, the cheek region located on both sides of the face from the eye to the lower jaw, the eye region composed of the outer edge of the eyelid, and the like.

[0062] In an optional embodiment, after obtaining the corresponding target face image based on the skin color information, the method further includes: performing smoothing processing on the target face image, and displaying the target face image on a display interface.

[0063] Specifically, the matching sub-region corresponding to each sub-region, especially the face region, is fused to obtain the target face image, and the noise or distortion on the target face image can be reduced by smooth (image smoothing). Finally, a high dynamic face image is output, which can obtain all the details of the face and match the brightness order of the portrait, so that the portrait is clear and stereoscopic. Then, the portrait is displayed on the display interface.

[0064] The original image obtained in the prior art may have bright and dark areas due to inappropriate exposure, and the forehead area of the obtained face may also have bright and dark areas in severe cases. The embodiment can effectively ensure that the obtained target image is clear and stereoscopic, and has good consistency, thereby improving the photographing experience of people with deep skin color and being suitable for processing face images in various scenes.

[0065] In combination with the above examples, reference is made to Figure 3 , Figure 3 The structure diagram of the face model provided in Embodiment Two of the present application is shown in the figure. The selection branch 21, the division branch 22, the matching branch 23, and the fusion branch 24 are specifically selected. The selection branch is used to select a face area from an original image to obtain a standard image. The division branch is used to extract features of the standard image, divide the standard image into a plurality of sub-areas, select a first area from the sub-areas, and obtain a plurality of candidate sub-areas corresponding to the sub-areas. The matching branch is used to match the features of the first area with the features of the candidate sub-areas to obtain at least one candidate sub-area as a matching sub-area. The skin color category corresponding to the first area is determined according to the matching sub-area. The fusion branch is used to obtain the matching sub-area corresponding to each sub-area in the face area according to the skin color category. All the matching sub-areas are fused to obtain a target face image.

[0066] In an optional embodiment, the original image is input into a skin color face model, the skin color face model can be divided into smaller sub-areas, and the obtained standard image can also be divided into the same number of smaller candidate sub-areas. Then, the features of the forehead area are matched with the corresponding candidate sub-areas to obtain at least one matched candidate sub-area, and then obtain a corresponding matching sub-area. The skin color category is determined according to the matching sub-area, for example, brown black. Then, the matching sub-area corresponding to other sub-areas is obtained according to the skin color category of brown black, that is, the matching sub-area is obtained from the plurality of candidate sub-areas corresponding to each sub-area, and all the matching sub-areas are fused to obtain a target face image. The target image can be effectively ensured to be clear and stereoscopic, and have good consistency, thereby improving the photographing experience of people with deep skin color and being suitable for processing face images in various scenes.

[0067] In an optional embodiment, before determining the skin color category corresponding to the first area according to the matching sub-area, the method further comprises: training a data set to obtain a one-to-one correspondence between the matching sub-area and the skin color category corresponding to the first area.

[0068] Specifically, the training data set includes a plurality of different skin color face images, which can be set and stored according to an existing face database, or can be collected and downloaded from the Internet to form a training data set. Based on a large number of deep skin color portrait analysis, a deep skin color N large class of people can be obtained, and the skin color classification is mainly divided according to the forehead region of the portrait as the reference point, because the forehead region contains the most information, such as containing H (Hue, hue or tone), S (Saturation, saturation), Y (yellow) value and the like.

[0069] Through multiple training of the training data set, a one-to-one correspondence relationship between the matching sub-region and the forehead region of the face corresponding to the skin color category is finally obtained, and has high stability and good robustness, which can ensure that the portrait is bright and will not be blurred in the backlight scene, and even if the original image is taken in the case of glare in the outdoor sunny day, the portrait effect can still be improved through the model.

[0070] The embodiment improves the dynamic range of the face in portrait shooting, especially for deep skin color portrait shooting, which can not only obtain clearer portrait details in various scenes, but also can perform three-dimensional portrait of deep skin color people, and can ensure consistency of portrait shooting in different environments.

[0071] Figure 4 The structure diagram of the face image processing device provided in the third embodiment of the present application is shown in Figure 4 The face image processing device of the present embodiment can include:

[0072] The acquisition module 33 is configured to acquire an original image.

[0073] The input module 34 is configured to input the original image as an input of a skin color face model. The skin color face model refers to: dividing the original image into skin color categories according to the forehead region of the face, and obtaining a corresponding target face image based on the skin color category.

[0074] The output module 35 is configured to output the target face image through the skin color face model.

[0075] The face image processing device of the present embodiment can execute the technical solutions in the method shown in Figure 2 The specific implementation process and technical principles are described in the related description of the method shown in Figure 2 The specific implementation process and technical principles are described in the related description of the method shown in

[0076] Figure 5 The structure diagram of the face image processing device provided in the fourth embodiment of the present application is shown in Figure 5 The face image processing device in the present embodiment can be used in Figure 4The original image is taken as an input of the skin color face model on the basis of the original image, and the method further comprises the following steps:

[0077] The constructing module 31 is configured to construct an initial face model, wherein the initial face model comprises a selection branch, a division branch, a matching branch and a fusion branch; the selection branch is configured to select a face region according to the original image to obtain a standard image; the division branch is configured to extract a feature of the standard image, divide the standard image into a plurality of sub-regions, select a first region from the sub-regions, and obtain a plurality of candidate sub-regions corresponding to the sub-regions; the matching branch is configured to match the feature of the first region with features of the candidate sub-regions to obtain at least one candidate sub-region corresponding to the first region as a matching sub-region; and the fusion branch is configured to obtain the matching sub-region corresponding to each sub-region in the face region according to the skin color category, and fuse all the matching sub-regions to obtain a target face image.

[0078] The obtaining module 32 is configured to train the initial face model through a training data set to obtain the skin color face model.

[0079] Figure 6 A structure diagram of the face image processing system provided by the fifth embodiment of the present application is shown in Figure 6 The face image processing system 40 of the present embodiment can comprise a processor 41 and a memory 42.

[0080] The memory 42 is configured to store computer programs (such as application programs and function modules for implementing the above-mentioned face image processing method), computer instructions and the like.

[0081] The above-mentioned computer programs, computer instructions and the like can be stored in one or more memories 42 in a partitioned manner. And the above-mentioned computer programs, computer instructions, data and the like can be called by the processor 41.

[0082] The processor 41 is configured to execute the computer programs stored in the memory 42 to implement each step in the method involved in the above-mentioned embodiments.

[0083] For details, please refer to the related description in the above method embodiments.

[0084] The processor 41 and the memory 42 can be an independent structure or an integrated structure. When the processor 41 and the memory 42 are an independent structure, the memory 42 and the processor 41 can be coupled and connected through a bus 43.

[0085] The server of the present embodiment can execute the technical solutions in the method shown in Figure 2 For details of the implementation process and technical principles, please refer to the related description in the method shown in Figure 2 The above-mentioned computer programs, computer instructions and the like can be stored in one or more memories 42 in a partitioned manner. And the above-mentioned computer programs, computer instructions, data and the like can be called by the processor 41.

[0086] In addition, the embodiments of the present application further provide a computer readable storage medium, which stores computer execution instructions. When at least one processor of a user equipment executes the computer execution instructions, the user equipment executes the various possible methods.

[0087] The computer readable medium includes a computer storage medium and a communication medium. The communication medium includes any medium that facilitates transfer of a computer program from one place to another. A storage medium can be any available medium that can be accessed by a general purpose or special purpose computer. An exemplary computer program product couples to a processor, such that the processor can read information from, and write information to, the computer program product. Of course, the computer program product can be the processor itself. By way of example, and not limitation, such computer program products include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired computer program code in the form of computer-executable instructions or data structures and that can be accessed by a general purpose or special purpose computer.

[0088] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by relevant hardware instructed by programs. The programs can be stored in a computer readable storage medium. When the programs are executed, the steps of the above-mentioned method embodiments are executed; and the storage medium includes ROM, RAM, magnetic or optical disk, and other media that can store program codes.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part or all of the technical features; and the modification or replacement does not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of face image processing, characterized by, The method comprises the following steps: obtaining an original image and selecting a face region from the original image; detecting that the brightness of the face region is not less than a preset target value, taking the corresponding original image as a standard image, and storing a current exposure T; extracting features of the standard image, dividing the standard image into a plurality of sub-regions, and selecting a forehead region of the face as a first region from the sub-regions; performing positive exposure and negative exposure processing on each of the sub-regions according to the current exposure T to obtain a plurality of candidate sub-regions; matching the features of the first region with the features of the candidate sub-regions to obtain at least one corresponding candidate sub-region as a matching sub-region; determining a skin color category corresponding to the first region according to the matching sub-region; obtaining a matching sub-region corresponding to each sub-region in the face region according to the skin color category; fusing all the matching sub-regions to obtain a target face image.

2. The method of claim 1, wherein, Before determining the skin color category corresponding to the first region according to the matching sub-region, the method further comprises the following steps: training through a training data set to obtain a one-to-one correspondence between the matching sub-region and the skin color category corresponding to the first region.

3. The method according to claim 1 or 2, characterized in that, The method of obtaining the original image comprises the following steps: detecting a face in a preview interface, if no face region is detected, entering a shooting mode, and performing shooting dimming according to a preset light measurement mode to obtain the original image.

4. An apparatus for face image processing, the apparatus comprising: The method comprises the following steps: an acquisition module is configured to obtain an original image and select a face region from the original image, detect that the brightness of the face region is not less than a preset target value, take the corresponding original image as a standard image, and store a current exposure T; an input module is configured to take the original image as an input of a skin color face model; wherein the skin color face model refers to the following steps: extracting features of the standard image, dividing the standard image into a plurality of sub-regions, and selecting a forehead region of the face as a first region from the sub-regions; performing positive exposure and negative exposure processing on each of the sub-regions according to the current exposure T to obtain a plurality of candidate sub-regions; matching the features of the first region with the features of the candidate sub-regions to obtain at least one corresponding candidate sub-region as a matching sub-region; determining a skin color category corresponding to the first region according to the matching sub-region; obtaining a matching sub-region corresponding to each sub-region in the face region according to the skin color category; and fusing all the matching sub-regions to obtain a target face image; an output module is configured to output the target face image through the skin color face model.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by a processor to implement the face image processing method of any one of claims 1-3.

Images