A method, device and computer storage medium for processing face images

Abstract

The present invention discloses a facial image processing method, device and computer storage medium. The method comprises the following steps: pre-processing a media stream to obtain an original media stream of a flashing star background layer; performing visual image processing on the original media stream of the flashing star background layer to obtain a first input texture; performing corner point information processing on the original media stream of the flashing star background layer to obtain a second input texture; performing effect superposition processing on the first input texture and the second input texture to obtain a media stream after facial processing; obtaining corner point positions according to picture / video materials and performing a human-scene separation operation, merging the results and performing an edge processing operation to ensure that the rendered effect is stable and accurate; using facial recognition technology to ensure that flash points do not appear on the face of a person, and controlling the number of flash points through a threshold granularity.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a face image processing method, device and computer storage medium. Background Art

[0002] At present, based on OpenGL ES graphics and imaging processing, GPU rendering processing is used for image / video materials. In order to provide efficient and accurate precision calculation and high-standard effect display, the graphics and imaging operation knowledge is used to process the vertex buffer of the material, load it into the vertex shader, and assemble the processed results through primitives. After a series of rasterization processes, the result set is read into the fragment shader according to the passed-in material texture, and finally presented on the screen.

[0003] In the prior art, there are slight differences in the results of each frame of the material transmitted by the input source, which causes the corner point detection results to change continuously, resulting in movement offset and stroboscopic phenomena in the drawn points; for face recognition detection, there is an inability to better recognize specific angles of the person's face, resulting in the effect not meeting the expected results; the processing method of the result image of human-scene separation is unreasonable, resulting in high memory usage during the entire rendering process, and sharpening of the edges between the person and the background. Summary of the invention

[0004] The present invention aims to solve the technical problems existing in the background technology and provides a face image processing method, device and computer storage medium.

[0005] In order to solve the technical problem, the technical solution of the present invention is:

[0006] A method for processing a face image, the method comprising:

[0007] The media stream is preprocessed to obtain an original media stream of the twinkling star background layer; the original media stream of the twinkling star background layer does not contain a grayscale media stream of the character layer.

[0008] The original media stream of the twinkling star background layer is processed by visual image processing to obtain a first input texture; the first input texture is a reference texture of the hybrid filter and is set in the zeroth layer of the hybrid filter;

[0009] The original media stream of the twinkling star background layer is processed with corner information to obtain a second input texture; the second input texture is the background texture of the hybrid filter and is set in the first layer of the hybrid filter;

[0010] Effect superposition processing is performed on the first input texture and the second input texture to obtain a media stream after face processing.

[0011] Further, the preprocessing specifically includes:

[0012] Performing human-scene separation processing on the media stream to obtain a layered grayscale media stream;

[0013] Using the preset twinkling star media stream, the layered grayscale media stream is overlaid to obtain the twinkling star background layer media stream;

[0014] The media stream of the twinkling star background layer and the layered grayscale media stream are masked to obtain the original media stream of the twinkling star background layer.

[0015] Furthermore, the media stream includes: picture materials and video materials; the layered grayscale media stream includes: scene layer grayscale media stream, background layer grayscale media stream and character layer grayscale media stream.

[0016] Furthermore, the preset twinkling star media stream is used to perform superposition processing on the background layer grayscale media stream to obtain the twinkling star background layer media stream.

[0017] Furthermore, mask processing is performed on the media stream of the twinkling star background layer, the grayscale media stream of the scenery layer and the grayscale media stream of the character layer to obtain the original media stream of the twinkling star background layer.

[0018] Furthermore, the original media stream of the twinkling star background layer is processed through OpenGL ES to obtain a first input texture.

[0019] Furthermore, the corner point information processing specifically includes:

[0020] The original media stream of the twinkling star background layer is subjected to the flash point detection by the Harris algorithm to obtain the Harris detection media stream;

[0021] Perform edge detection on the Harris detection media stream to obtain edge point detection media stream;

[0022] Perform slight blur processing on the edge point detection media stream to obtain the point media stream with blur effect;

[0023] The point media stream with blur effect is processed by corner detection to obtain the media stream after face filtering;

[0024] The media stream after face filtering is rendered to obtain a second input texture.

[0025] Furthermore, the corner point detection process specifically includes:

[0026] For the point media stream with blur effect, a threshold is set and face point area is filtered to obtain the media stream after face filtering.

[0027] A facial image processing device, comprising:

[0028] A layering module is used to perform a human-scene separation process on the media stream to obtain a layered grayscale media stream;

[0029] The superposition module is used to use the preset twinkling star media stream to perform superposition processing on the layered grayscale media stream to obtain the twinkling star background layer media stream;

[0030] The mask module is used to perform mask processing on the flash star background layer media stream and the layered grayscale media stream to obtain the original media stream of the flash star background layer;

[0031] The first processing module is used for obtaining a first input texture by visual image processing on the original media stream of the twinkling star background layer;

[0032] The second processing module processes the corner information of the original media stream of the twinkling star background layer to obtain a second input texture;

[0033] The third processing module performs effect superposition processing on the first input texture and the second input texture to obtain a media stream after face processing.

[0034] A computer-readable storage medium stores computer-executable instructions, which are used to implement any of the above methods when executed by a processor.

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] The corner positions are obtained based on the image / video material and the human-scene separation operation is performed. The results are merged and processed at the edge to ensure that the rendering effect is stable and accurate. Face recognition technology ensures that flash points do not appear on the face of the person, and the number of flash points is controlled through the threshold granularity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 , a main flow chart of a face image processing method of the present invention;

[0038] Figure 2 , a preprocessing flow chart of a face image processing method of the present invention;

[0039] Figure 3 , a flowchart of corner point information processing of a face image processing method of the present invention;

[0040] Figure 4 , schematic diagram of Harris algorithm;

[0041] Figure 5 , images containing people before and after preprocessing and processing;

[0042] Figure 6, black and white image after the people-scene separation operation;

[0043] Figure 7 , the effect diagram after corner point information processing. DETAILED DESCRIPTION

[0044] The specific implementation mode of the present invention is described below in conjunction with embodiments:

[0045] It should be noted that the structures, proportions, sizes, etc. shown in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0046] At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0047] Embodiment 1:

[0048] like Figure 1 As shown, a facial image processing method comprises:

[0049] The media stream is preprocessed to obtain an original media stream of the twinkling star background layer; the original media stream of the twinkling star background layer does not contain a grayscale media stream of the character layer.

[0050] The original media stream of the twinkling star background layer is processed by visual image processing to obtain a first input texture; the first input texture is a reference texture of the hybrid filter and is set in the zeroth layer of the hybrid filter;

[0051] The original media stream of the twinkling star background layer is processed with corner information to obtain a second input texture; the second input texture is the background texture of the hybrid filter and is set in the first layer of the hybrid filter;

[0052] Effect superposition processing is performed on the first input texture and the second input texture to obtain a media stream after face processing.

[0053] Further, the preprocessing specifically includes:

[0054] Performing human-scene separation processing on the media stream to obtain a layered grayscale media stream;

[0055] Using the preset twinkling star media stream, the layered grayscale media stream is overlaid to obtain the twinkling star background layer media stream;

[0056] The media stream of the twinkling star background layer and the layered grayscale media stream are masked to obtain the original media stream of the twinkling star background layer.

[0057] Furthermore, the media stream includes: picture materials and video materials; the layered grayscale media stream includes: scene layer grayscale media stream, background layer grayscale media stream and character layer grayscale media stream.

[0058] Furthermore, the preset twinkling star media stream is used to perform superposition processing on the background layer grayscale media stream to obtain the twinkling star background layer media stream.

[0059] Furthermore, mask processing is performed on the media stream of the twinkling star background layer, the grayscale media stream of the scenery layer and the grayscale media stream of the character layer to obtain the original media stream of the twinkling star background layer.

[0060] Furthermore, the original media stream of the twinkling star background layer is processed through OpenGL ES to obtain a first input texture.

[0061] Furthermore, the corner point information processing specifically includes:

[0062] The original media stream of the twinkling star background layer is subjected to the flash point detection by the Harris algorithm to obtain the Harris detection media stream;

[0063] Perform edge detection on the Harris detection media stream to obtain edge point detection media stream;

[0064] Perform slight blur processing on the edge point detection media stream to obtain the point media stream with blur effect;

[0065] The point media stream with blur effect is processed by corner detection to obtain the media stream after face filtering;

[0066] The media stream after face filtering is rendered to obtain a second input texture.

[0067] Furthermore, the corner point detection process specifically includes:

[0068] For the point media stream with blur effect, a threshold is set and face point area is filtered to obtain the media stream after face filtering.

[0069] A facial image processing device, comprising:

[0070] A layering module is used to perform a human-scene separation process on the media stream to obtain a layered grayscale media stream;

[0071] The superposition module is used to use the preset twinkling star media stream to perform superposition processing on the layered grayscale media stream to obtain the twinkling star background layer media stream;

[0072] The mask module is used to perform mask processing on the flash star background layer media stream and the layered grayscale media stream to obtain the original media stream of the flash star background layer;

[0073] The first processing module is used for obtaining a first input texture by visual image processing on the original media stream of the twinkling star background layer;

[0074] The second processing module processes the corner information of the original media stream of the twinkling star background layer to obtain a second input texture;

[0075] The third processing module performs effect superposition processing on the first input texture and the second input texture to obtain a media stream after face processing.

[0076] A computer-readable storage medium stores computer-executable instructions, which are used to implement any of the above methods when executed by a processor.

[0077] Compared with the prior art, the advantages of the present invention are:

[0078] The corner positions are obtained based on the image / video material and the human-scene separation operation is performed. The results are merged and processed at the edge to ensure that the rendering effect is stable and accurate. Face recognition technology ensures that flash points do not appear on the face of the person, and the number of flash points is controlled through the threshold granularity.

[0079] Embodiment 2:

[0080] like Figure 2 As shown, after the system collects the picture / video material, it uses the person-scene separation algorithm to convert the picture into a black-and-white picture. According to the difference in black and white colors in the black-and-white picture, the scene layer, background layer and person layer are processed separately. For the background layer, the mixed RGBA algorithm is used to superimpose the required flashing star effect material on the background layer, and the background layer containing the flashing star effect is obtained. The mask operation is performed with the original material to obtain the operation only for the background layer, and the processing result is used as the input source of the flashing star effect; it is passed to Figure 3 In the process shown.

[0081] like Figure 3 As shown in the figure, after the system collects the image / video material of people-scene separation, it is processed into the first input texture through Open GLES, which is used as the reference texture of the hybrid filter. At the same time, the material of the first texture is processed through flash point detection, edge detection, slight blur processing, corner point detection, and flash drawing in sequence as the second input texture. Finally, the first input texture is mixed with the second input texture by RGBA, and passed to the next node as the input source of the screen display, and finally presented on the screen.

[0082] against Figure 3 In the process shown, the algorithm of corner point detection will filter the points obtained according to the face recognition area, select all points within the range according to the maximum and minimum boundaries of the face area, filter the points obtained from the corner point detection results and merge the points that are repeated nearby. Finally, the points that are not in the face position and are repeatedly processed by the merging algorithm are used as the point information for flash point drawing.

[0083] During the rendering process, in order to improve the accuracy of face recognition points, the results of the previous frame are used to make up for the current undetected results to prevent the problem of a large number of flashing points on the face due to the failure to detect the face. At the same time, in order to ensure the memory consumption and frame rate loss during the face recognition process, the granularity specified for face scanning is the lowest, thus ensuring that the performance loss is maintained at a relatively stable level while achieving the effect of detecting the face.

[0084] The human-scene separation effect improves the connection between the human and the scenery. By setting the sharpening granularity to process the edge, and setting a relatively smooth edge granularity value, the human and the scenery can be more closely matched, ensuring the visual integration of the human and the scenery.

[0085] Corner detection algorithm (Harris algorithm):

[0086] like Figure 4 As shown, assuming that the coordinates of each pixel in the current window are (xi, yj), where (x, y)∈W, when we offset the window by (u, v) pixels, we define the weighted sum of squared differences (weighted summed square difference, abbreviated as SSD).

[0087] E(u,v)=∑ω(x,y)[I(x+u,y+v)-I(x,y)]2((x,y)∈W).

[0088] Where ω(x,y) represents the weight of each pixel. The commonly used weight kernel is the Gaussian kernel, which can also be set to all 1.

[0089] Realization of people-scene separation effect:

[0090] Figure 5 It shows the images containing people that need to be preprocessed and processed before, and obtains the image through the person-scene separation operation. Figure 6 In the black and white picture shown, the white is the background layer and the black is the extracted character layer. By operating the extracted result image and superimposing the result image on the original image, we can get what we expect. Figure 7 Effect.

[0091] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the pending claims of the invention.

Claims

1. A facial image processing method, characterized in that: The method comprises: Preprocess the media stream to obtain the original media stream of the twinkling star background layer; The original media stream of the twinkling star background layer is processed by visual image processing to obtain a first input texture; Performing corner point information processing on the original media stream of the twinkling star background layer to obtain a second input texture; Performing effect superposition processing on the first input texture and the second input texture to obtain a media stream after face processing; The pre-processing specifically includes: Performing human-scene separation processing on the media stream to obtain a layered grayscale media stream; Using the preset twinkling star media stream, the layered grayscale media stream is overlaid to obtain the twinkling star background layer media stream; Performing mask processing on the media stream of the twinkling star background layer and the layered grayscale media stream to obtain the original media stream of the twinkling star background layer; The corner point information processing specifically includes: The original media stream of the twinkling star background layer is subjected to the flash point detection by the Harris algorithm to obtain the Harris detection media stream; Perform edge detection on the Harris detection media stream to obtain edge point detection media stream; Perform slight blur processing on the edge point detection media stream to obtain the point media stream with blur effect; The point media stream with blur effect is processed by corner detection to obtain the media stream after face filtering; The media stream after face filtering is rendered to obtain a second input texture; The corner point detection process specifically includes: For the point media stream with blur effect, a threshold is set and face point area is filtered to obtain the media stream after face filtering.

2. A facial image processing method according to claim 1, characterized in that: The media stream includes: picture materials and video materials; the layered grayscale media stream includes: scene layer grayscale media stream, background layer grayscale media stream and character layer grayscale media stream.

3. A facial image processing method according to claim 2, characterized in that: The preset twinkling star media stream is used to overlay the background layer grayscale media stream to obtain the twinkling star background layer media stream.

4. A facial image processing method according to claim 2, characterized in that: The media stream of the twinkling star background layer, the grayscale media stream of the scenery layer and the grayscale media stream of the character layer are subjected to mask processing to obtain the original media stream of the twinkling star background layer.

5. A facial image processing method according to claim 1, characterized in that: The original media stream of the twinkling star background layer is processed through OpenGL ES to obtain the first input texture.

6. A facial image processing device, characterized in that: include: A layering module is used to perform a human-scene separation process on the media stream to obtain a layered grayscale media stream; The superposition module is used to use the preset twinkling star media stream to perform superposition processing on the layered grayscale media stream to obtain the twinkling star background layer media stream; The mask module is used to perform mask processing on the flash star background layer media stream and the layered grayscale media stream to obtain the original media stream of the flash star background layer; The first processing module is used for obtaining a first input texture by visual image processing on the original media stream of the twinkling star background layer; The second processing module processes the corner information of the original media stream of the twinkling star background layer to obtain a second input texture; A third processing module performs effect superposition processing on the first input texture and the second input texture to obtain a media stream after face processing; Preprocessing specifically includes: Performing human-scene separation processing on the media stream to obtain a layered grayscale media stream; Using the preset twinkling star media stream, the layered grayscale media stream is overlaid to obtain the twinkling star background layer media stream; Performing mask processing on the media stream of the twinkling star background layer and the layered grayscale media stream to obtain the original media stream of the twinkling star background layer; The corner point information processing specifically includes: The original media stream of the twinkling star background layer is subjected to the flash point detection by the Harris algorithm to obtain the Harris detection media stream; Perform edge detection on the Harris detection media stream to obtain edge point detection media stream; Perform slight blur processing on the edge point detection media stream to obtain the point media stream with blur effect; The point media stream with blur effect is processed by corner detection to obtain the media stream after face filtering; The media stream after face filtering is rendered to obtain a second input texture; The corner point detection process specifically includes: For the point media stream with blur effect, a threshold is set and face point area is filtered to obtain the media stream after face filtering.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

Images