A face image processing method, device, equipment and storage medium

Abstract

The present disclosure provides a face image processing method, device and equipment and a storage medium. The method comprises the following steps: first, determining the identity of a first pixel point in a face region of a current frame face image; then, determining a preset operation corresponding to the identity of the first pixel point; finally, performing the preset operation on the first pixel point in the current frame face image. The present disclosure performs the preset operation on the first pixel point based on the identity reflecting whether the first pixel point has been blocked, enriches the function of the interactive application program, and improves the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and in particular to a method, apparatus, device and storage medium for processing facial images. Background Technology

[0002] In interactive applications, to attract users and increase user base, a common approach is to enrich the interactive features. For example, short video applications offer users the ability to add special effects while shooting short videos, such as adding animal ear effects or fox tail effects when shooting short videos of people.

[0003] However, due to the rapid updates and iterations of gameplay in interactive applications, how to implement features and gameplay that can attract users and thus improve the user experience of interactive applications has become a pressing technical problem for application developers. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a face image processing method, apparatus, device, and storage medium, which can enrich the functionality of interactive applications and improve the user experience.

[0005] In a first aspect, this disclosure provides a face image processing method, the method comprising: Determine the identifier of the first pixel in the face region of the current frame face image; wherein, the identifier of the first pixel is used to identify whether the first pixel is occluded in the current frame face image or at least one frame of face image in the previous n frames of the current frame face image; A preset operation is determined to correspond to the identifier of the first pixel. The preset operation is performed on the first pixel in the current frame face image.

[0006] In an optional implementation, before determining the identifier of the first pixel in the face region of the current frame face image, the method further includes: Obtain the identifier of the second pixel in the face region of the previous face image of the current frame face image; wherein the second pixel has a positional correspondence with the first pixel in the face region of the current frame face image, and the identifier of the second pixel is used to identify whether the second pixel is occluded in the previous face image of the current frame face image or in at least one face image in the m frames preceding the previous face image of the previous frame face image; Accordingly, determining the identifier of the first pixel in the face region of the current frame face image includes: Based on the identifier of the second pixel, the identifier of the first pixel in the face region of the current frame face image is determined.

[0007] In one optional implementation, determining the identifier of the first pixel in the face region of the current frame face image based on the identifier of the second pixel includes: If the identifier of the second pixel is a preset first identifier, then the value of the preset first identifier is determined as the identifier of the first pixel; wherein, the preset first identifier is used to identify that the second pixel is in an occluded state in at least one frame of face image in the previous frame or in the m frames preceding the previous frame of face image; or, If the identifier of the second pixel is a preset second identifier, then the identifier of the first pixel is determined based on whether the first pixel in the face region of the current frame face image is in the face occlusion region of the current frame face image; wherein, the preset second identifier is used to indicate that the second pixel is not in the occlusion state in the previous frame face image and the face images before the previous m frames of the previous frame face image.

[0008] In one optional implementation, determining the identifier of the first pixel based on whether it is located within a face occlusion area of ​​the current frame face image includes: The current frame face image is input into the face occlusion detection model. After processing by the face occlusion detection model, the face occlusion area on the current frame face image is output. Determine whether the first pixel in the face region of the current frame face image is located in the face occlusion area; If it is determined that the first pixel is located in the face occlusion area, then the value of the preset first identifier is determined as the identifier of the first pixel; otherwise, the value of the preset second identifier is determined as the identifier of the first pixel.

[0009] In one optional implementation, performing the preset operation on the first pixel in the current frame face image includes: If the identifier of the first pixel is the preset first identifier, then perform the first preset operation on the first pixel; or, If the identifier of the first pixel is the preset second identifier, then the second preset operation is performed on the first pixel.

[0010] In one optional implementation, determining the identifier of the first pixel in the face region of the current frame face image includes: The current frame face image is input into the face occlusion detection model. After processing by the face occlusion detection model, the face occlusion area on the current frame face image is output. The occlusion region of the face in the current frame image is deformed to a standard face to obtain the deformed occlusion region; The identifier of the first pixel is determined based on whether the third pixel on the standard face, which has a positional correspondence with the first pixel on the current frame face image, is located in the deformed face occlusion area and the identifier of the third pixel; wherein, the identifier of the third pixel is updated based on the identifier of the second pixel on the previous frame face image, which has a positional correspondence with the third pixel.

[0011] In an optional implementation, after determining the identifier of the first pixel, the method further includes: Based on the identifier of the first pixel, update the identifier of the third pixel on the standard face.

[0012] In one optional implementation, the preset operation of determining the correspondence between the identifier of the first pixel includes: A preset operation is determined to have a corresponding relationship with the identifier of the third pixel.

[0013] In one optional implementation, determining the identifier of the first pixel in the face region of the current frame face image includes: The identifier of the first pixel is determined based on whether the first pixel in the face region of the current frame face image is located in the face occlusion area of ​​the current frame face image.

[0014] Secondly, this disclosure provides a face image processing apparatus, the apparatus comprising: The first determining module is used to determine the identifier of a first pixel in the face region of the current frame face image; wherein, the identifier of the first pixel is used to indicate whether the first pixel is occluded in the current frame face image or at least one frame of face image in the previous n frames of the current frame face image; The second determining module is used to determine a preset operation that corresponds to the identifier of the first pixel. An execution module is used to perform the preset operation on the first pixel point in the current frame face image.

[0015] Thirdly, this disclosure provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the above-described method.

[0016] Fourthly, this disclosure provides an apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described above.

[0017] Fifthly, this disclosure provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the method described above.

[0018] The technical solution provided in this disclosure has the following advantages compared with the prior art: This disclosure provides a face image processing method. First, an identifier for a first pixel in the face region of the current frame face image is determined. Then, a preset operation corresponding to the identifier of the first pixel is determined. Finally, the preset operation is performed on the first pixel in the current frame face image. This disclosure, based on an identifier indicating whether the first pixel has been occluded, implements the preset operation on the first pixel, enriching the functionality of interactive applications and improving the user experience. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a face image processing method provided in this disclosure embodiment; Figure 2 A schematic diagram of a current frame face image provided in an embodiment of this disclosure; Figure 3 A flowchart of another face image processing method provided in this disclosure embodiment; Figure 4 A flowchart of another face image processing method provided in this disclosure embodiment; Figure 5 This is a schematic diagram of the structure of a face image processing device provided in an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of a face image processing device provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0023] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0024] Interactive applications typically employ a variety of gameplay elements to attract users and increase their user base. Therefore, how to enhance the gameplay of interactive applications has become a question that application developers are constantly exploring.

[0025] For example, short video applications already offer users the ability to add special effects during video recording, such as animal ear effects or fox tail effects when shooting videos of people. How to further enrich the variety of special effects and enhance the user experience is a pressing technical challenge that needs to be addressed.

[0026] To address this, this disclosure provides a face image processing method. First, an identifier for a first pixel in the face region of the current frame face image is determined. Then, a preset operation corresponding to the identifier of the first pixel is determined. Finally, the preset operation is displayed on the current frame face image for the first pixel. This embodiment of the disclosure, based on an identifier indicating whether the first pixel has been occluded, implements the execution of the preset operation for the first pixel, enriching the functionality of interactive applications and improving the user experience.

[0027] Based on this, the present disclosure provides a face image processing method, referring to... Figure 1 The flowchart below illustrates a face image processing method provided in this embodiment of the present disclosure. The method includes: S101: Determine the identifier of the first pixel in the face region of the current frame face image.

[0028] The identifier of the first pixel is used to indicate whether the first pixel is occluded in the face image of the current frame or in at least one of the face images of the previous n frames. The previous n frames can be the previous frame.

[0029] In this embodiment of the disclosure, the current frame face image can be the face image currently being displayed in the video stream. For example, when a user is taking a picture with the camera, the current frame face image can be an image region containing a face in the image currently captured by the camera. As another example, when a user is taking a selfie with the front-facing camera, the current frame face image is an image region of a preset size containing a face in the current user image captured by the camera, such as 256 pixels. A 256-size image region containing a human face.

[0030] like Figure 2 The above is a schematic diagram of a current frame face image provided in an embodiment of this disclosure, wherein the image area within the rectangle is the current frame face image in this embodiment of the disclosure.

[0031] In practical applications, after obtaining the face image of the current frame, the face region on the face image of the current frame is further determined.

[0032] In one optional implementation, a face detection network can be used to process the current frame face image to obtain facial key points. Then, the face region on the current frame face image can be determined based on the facial key points. Specifically, the face detection network is pre-trained using a large number of face image samples labeled with facial key points to obtain a trained face detection network. Then, the current frame face image is input into the face detection network, processed, and the facial key points on the current frame face image are output.

[0033] like Figure 2 As shown, the black dots exemplarily represent some facial key points, and the closed area formed by connecting these facial key points in sequence is the face region.

[0034] In this embodiment of the disclosure, a preset operation is performed on each pixel in the face region of the current frame face image. The first pixel can be any pixel in the face region of the current frame face image.

[0035] In addition, in this embodiment of the present disclosure, the identifier of the first pixel is used to characterize whether the first pixel has been occluded, specifically to identify whether the first pixel is in an occluded state in the face image of the current frame or in at least one face image of the previous n frames of the face image of the current frame.

[0036] In one optional implementation, the identifier of the first pixel can be determined by combining whether the first pixel is occluded in the current frame of the face image and the identifier of a second pixel that has a positional correspondence with the first pixel in at least one of the previous n frames of the face image. That is, the identifier of the first pixel can not only reflect whether the first pixel is occluded in the current frame of the face image, but also whether pixels that have a positional correspondence with the first pixel in at least one of the previous n frames of the face image are occluded. Specific implementation methods will be described in subsequent embodiments.

[0037] In another alternative implementation, the identifier of the first pixel can be determined solely based on whether the first pixel is occluded in the current frame of the face image. Specifically, the identifier of the first pixel is determined based on whether the first pixel in the face region of the current frame of the face image is within an occluded area of ​​the face in the current frame of the face image. For example, when processing the first frame image, the identifier of the first pixel can be determined based on whether the first pixel in the face region of the first frame of the face image is within an occluded area of ​​the face in the current frame of the face image.

[0038] In practical applications, a face occlusion detection network can be used to process the current frame of a face image to obtain the occlusion regions on the face image. An occlusion region refers to the area on the face image that is obscured by other objects, such as a hand. Specifically, the face occlusion detection network is pre-trained using a large number of face image samples labeled with occlusion regions. Then, the current frame of the face image is input into this network, and after processing, the network outputs the occlusion regions on the current frame of the face image. For example... Figure 2 As shown, the area covered by the hand in the face image is the face occlusion area.

[0039] After determining the face occlusion region in the current frame face image, it is further determined whether a first pixel is located within the face occlusion region, and then the identifier of the first pixel is further determined based on the determination result. Specifically, if it is determined that the first pixel is located within the face occlusion region, the identifier of the first pixel can be determined as a preset first identifier, which indicates that the first pixel is in an occluded state in the current frame face image; if it is determined that the first pixel is not located within the face occlusion region, the identifier of the first pixel can be determined as a preset second identifier, which indicates that the first pixel is not in an occluded state in the current frame face image.

[0040] S102: Determine a preset operation that corresponds to the identifier of the first pixel.

[0041] In this embodiment of the disclosure, a correspondence between an identifier and a preset operation is established in advance. After determining the identifier of the first pixel, the preset operation corresponding to the identifier is determined from the correspondence between the identifier and the preset operation.

[0042] For example, the preset operation can be a preset effect. Assume the identifiers include a preset first identifier and a preset second identifier, where the preset first identifier corresponds to the "charcoal face" effect, and the preset second identifier corresponds to the "beautiful face" effect. If the identifier of the first pixel is the preset first identifier, then the "charcoal face" effect corresponding to the preset first identifier is determined; if the identifier of the first pixel is the preset second identifier, then the "beautiful face" effect corresponding to the preset second identifier is determined.

[0043] The embodiments disclosed herein do not limit the setting of the correspondence between identifiers and preset operations, and can be determined based on face processing requirements.

[0044] S103: Perform the preset operation on the first pixel in the current frame face image.

[0045] In this embodiment of the disclosure, after determining the preset operation corresponding to the identifier of the first pixel, the preset operation is performed on the first pixel.

[0046] For example, if the preset operation is a preset effect, then after determining that the effect corresponding to the identifier of the first pixel is the "charcoal face" effect, the "charcoal face" effect will be displayed on the face image of the current frame for that first pixel.

[0047] In the face image processing method provided in this embodiment, the identifier of the first pixel in the face region of the current frame face image is first determined. Then, a preset operation corresponding to the identifier of the first pixel is determined. Finally, the preset operation is performed on the first pixel in the current frame face image. This embodiment, based on an identifier reflecting whether the first pixel has been occluded, implements the execution of the preset operation on the first pixel, enriching the functionality of interactive applications and improving the user experience.

[0048] Based on the above embodiments, this disclosure also provides a face image processing method, referring to... Figure 3 This is a flowchart of another face image processing method provided in this disclosure embodiment, the method including: S301: Obtain the identifier of the second pixel in the face region of the previous frame face image of the current frame face image.

[0049] Wherein, the second pixel has a positional correspondence with the first pixel in the face region of the current frame face image, and the identifier of the second pixel is used to identify whether the second pixel is in an occluded state in the previous frame face image of the current frame face image or in at least one frame face image in the m frames preceding the previous frame face image.

[0050] In one optional implementation, the occlusion region of the face in the previous frame of the current frame is deformed into a standard face, wherein a third pixel on the standard face has a positional correspondence with a first pixel on the current frame. Then, a second pixel on the previous frame that has a positional correspondence with the third pixel on the standard face is determined, and its identifier is determined based on the identifier of the third pixel on the standard face and whether the third pixel is within the occlusion region of the previous frame. The standard face is a pre-set, unoccluded face image of standard size. Specifically, the occlusion region of the face in the previous frame can be deformed into a standard face based on facial key points; the specific implementation method is not limited in this embodiment.

[0051] In practical applications, when processing the current frame of a face image, it is sufficient to obtain the identifier of the second pixel point that has a positional correspondence with the first pixel point in the current frame of the face image, which is already determined in the previous frame of the face image.

[0052] S302: Based on the identifier of the second pixel, determine the identifier of the first pixel in the face region of the current frame face image.

[0053] In this embodiment of the disclosure, after obtaining the identifier of a second pixel that has a positional correspondence with a first pixel in the face region of the previous frame of the current frame face image, the identifier of the first pixel in the face region of the current frame face image is determined based on the identifier of the second pixel. Specifically, the identifier of the second pixel indicates whether the second pixel was previously occluded. Specifically, the identifier of the second pixel is used to indicate whether the second pixel was occluded in the previous frame of the current frame face image or in at least one of the m frames preceding the previous frame face image. Here, m ≤ 10.

[0054] In this embodiment of the present disclosure, when it is determined that the identifier of the second pixel indicates that the second pixel is in an occluded state, it can be determined that the identifier of the first pixel in the face region of the current frame face image that has a positional correspondence with the second pixel is also set as an identifier for indicating that it is in an occluded state.

[0055] Furthermore, when it is determined that the identifier of the second pixel indicates that the second pixel is not occluded, the identifier of the first pixel can be set based on whether the first pixel, which has a positional correspondence with the second pixel in the face region of the current frame face image, is occluded. Specifically, if the first pixel in the face region of the current frame face image is occluded, then the identifier of the first pixel is set to indicate that the first pixel is occluded; otherwise, the identifier of the first pixel is set to indicate that the first pixel is not occluded.

[0056] In one optional implementation, a preset first identifier is used to identify whether the second pixel is occluded in the previous frame of the face image in the current frame or in at least one of the m frames preceding the previous frame of the face image. If the identifier of the second pixel is determined to be the preset first identifier, then the value of the preset first identifier is determined to be the identifier of the first pixel in the current frame of the face image that has a positional correspondence with the second pixel.

[0057] A preset second identifier is used to identify that the second pixel is not occluded in the previous frame of the face image and in at least one of the m frames preceding the previous frame. If the identifier of the second pixel is determined to be the preset second identifier, then the identifier of the first pixel is determined based on whether the first pixel in the face region of the current frame face image, which has a positional correspondence with the second pixel, is located in the face occlusion region of the current frame face image.

[0058] Specifically, if it is determined that the first pixel is located in the face occlusion area of ​​the current frame face image, then the value of the preset first identifier is determined as the identifier of the first pixel; otherwise, the value of the preset second identifier is determined as the identifier of the first pixel.

[0059] S303: Determine a preset operation that corresponds to the identifier of the first pixel.

[0060] S304: Perform the preset operation on the first pixel in the current frame face image.

[0061] S303 and S304 in this embodiment can be understood with reference to S102 and S103 in the above embodiment, and will not be repeated here.

[0062] In the face image processing method provided in this embodiment, the identifier of the first pixel is determined by combining whether the first pixel in the current frame face image is occluded and whether the second pixel in at least one of the previous n frames face images that has a positional correspondence with the first pixel is occluded. This enables the execution of a preset operation on the first pixel, enriches the functionality of interactive applications, and improves the user experience.

[0063] Based on the above embodiments, this disclosure provides an optional implementation of a face image processing method. Specifically, firstly, the current frame face image is input into a face occlusion detection model. After processing by the face occlusion detection model, the face occlusion region on the current frame face image is output. Then, the face occlusion region of the current frame face image is deformed to a standard face to obtain the deformed face occlusion region. Next, based on whether a third pixel on the standard face, which has a positional correspondence with a first pixel on the current frame face image, is located within the deformed face occlusion region and the identifier of the third pixel, the identifier of the first pixel is determined. The identifier of the third pixel is updated based on the identifier of a second pixel on the previous frame face image, which has a positional correspondence with the third pixel.

[0064] In this embodiment of the disclosure, after determining the identifier of the first pixel, the identifier of the third pixel on the standard face is updated based on the identifier of the first pixel. Then, a preset operation corresponding to the identifier of the third pixel is determined, and the preset operation is performed on the first pixel in the current frame face image.

[0065] Based on the above, the application of the facial image processing method provided in this disclosure to short video applications can be used as an example to provide a more detailed description of the optional implementation methods. Specifically, during the shooting of short videos, special effects can be achieved by processing each frame of the facial image.

[0066] refer to Figure 4 , Figure 4 A flowchart of another face image processing method provided in this disclosure embodiment, wherein the method includes: S401: Pre-initialize the size of the standard human face mask model (mask) to 256. 256, and the identifier of each pixel on the standard face mask is 1.

[0067] In this standard face mask, each pixel is marked as 1, indicating that none of the pixels on the standard face are obscured. In this embodiment, mask_0 represents the initialized standard face mask.

[0068] S402: Upon receiving the first frame of a face image, determine the face occlusion area on the first frame of the face image, and deform the face occlusion area into the standard face to obtain the deformed face occlusion area.

[0069] S403: Based on the deformed face occlusion area, update the identifier of each pixel in mask_0 to obtain mask_1.

[0070] Specifically, the identifiers of pixels in the deformed face occlusion area of ​​mask_0 are updated to 0, indicating that these pixels have been occluded, thus obtaining mask_1.

[0071] S404: Based on the identifiers of each pixel in mask_1, display special effects on the first frame of the face image.

[0072] S405: Upon receiving the second frame of a face image, determine the face occlusion area on the second frame of the face image, and deform the face occlusion area into the standard face to obtain the deformed face occlusion area.

[0073] S406: Based on the deformed face occlusion area, update the identifier of each pixel in mask_1 to obtain mask_2.

[0074] Specifically, for pixels in mask_1 that are located within the deformed face occlusion region, the process first determines whether the pixel's identifier in mask_1 is 0. If it is, it means the pixel was occluded in the previous frame of the face image, and the identifier in mask_2 is directly set to 0, without considering the occlusion status of the pixel in the current frame. If the pixel's identifier is 1, it means the pixel was not occluded in the previous frame, and the identifier is set based on the occlusion status of the pixel in the current frame. In detail, if the pixel is within the deformed face occlusion region, the identifier in mask_2 is set to 0; otherwise, it is set to 1.

[0075] S407: Based on the identifiers of each pixel in mask_2, display special effects on the second frame of the face image.

[0076] S408: Based on the above method, display special effects sequentially on each frame of face image.

[0077] The face image processing method provided in this disclosure processes each frame of face image captured by the user in a short video application to achieve special effects display, enriching the functionality of short video applications and improving the user experience.

[0078] Similar to the above-described method embodiments, this disclosure also provides a face image processing apparatus, see reference. Figure 5 This is a schematic diagram of the structure of a face image processing device provided in an embodiment of the present disclosure. The device includes: The first determining module 501 is used to determine the identifier of a first pixel in the face region of the current frame face image; wherein, the identifier of the first pixel is used to identify whether the face image of the current frame or at least one frame of face image in the previous n frames of the current frame face image is in an occluded state. The second determining module 502 is used to determine a preset operation that corresponds to the identifier of the first pixel. The display module 503 is used to perform the preset operation on the first pixel point in the current frame face image.

[0079] In one optional embodiment, the apparatus further includes: The third determining module is used to obtain the identifier of the second pixel in the face region of the previous face image of the current frame face image; wherein the second pixel has a positional correspondence with the first pixel in the face region of the current frame face image, and the identifier of the second pixel is used to identify whether the second pixel is in an occluded state in the previous face image of the current frame face image or in at least one face image in the m frames preceding the previous face image of the previous frame face image; Accordingly, the first determining module is specifically used for: Based on the identifier of the second pixel, the identifier of the first pixel in the face region of the current frame face image is determined.

[0080] In one optional implementation, the first determining module includes: A first determining submodule is configured to determine the value of the preset first identifier as the identifier of the first pixel when the identifier of the second pixel is a preset first identifier; wherein, the preset first identifier is used to identify that the second pixel is in an occluded state in at least one frame of the face image in the previous frame or in at least one of the m frames preceding the previous frame of the face image; or... The second determining submodule is used to determine the identifier of the first pixel point based on whether the first pixel point in the face region of the current frame face image is in the face occlusion region of the current frame face image when the identifier of the second pixel point is a preset second identifier; wherein, the preset second identifier is used to indicate that it is not in the occlusion state in the previous frame face image and the face images m frames before the previous frame face image.

[0081] In one optional implementation, the second determining submodule includes: The first occlusion detection submodule is used to input the current frame face image into the face occlusion detection model, and after processing by the face occlusion detection model, output the face occlusion area on the current frame face image. The third determining submodule is used to determine whether the first pixel in the face region of the current frame face image is located in the face occlusion region; The fourth determining submodule is used to determine the value of the preset first identifier as the identifier of the first pixel when it is determined that the first pixel is in the face occlusion area; otherwise, it determines the value of the preset second identifier as the identifier of the first pixel.

[0082] In one optional implementation, the execution module includes: A first execution submodule is configured to perform a first preset operation on the first pixel when the identifier of the first pixel is the preset first identifier; or... The second execution submodule is used to perform a second preset operation on the first pixel when the identifier of the first pixel is the preset second identifier.

[0083] In one optional implementation, the first determining module includes: The second occlusion detection submodule is used to input the current frame face image into the face occlusion detection model, and after processing by the face occlusion detection model, output the face occlusion area on the current frame face image. The deformation submodule is used to deform the face occlusion area of ​​the current frame face image to a standard face, so as to obtain the deformed face occlusion area; The fifth determining submodule is used to determine the identifier of the first pixel based on whether the third pixel on the standard face, which has a positional correspondence with the first pixel on the current frame face image, is located in the deformed face occlusion area and the identifier of the third pixel; wherein, the identifier of the third pixel is updated based on the identifier of the second pixel on the previous frame face image, which has a positional correspondence with the third pixel.

[0084] In one optional embodiment, the apparatus further includes: An update module is used to update the identifier of the third pixel on the standard face based on the identifier of the first pixel.

[0085] In one optional implementation, the second determining module is specifically used for: A preset operation is determined to have a corresponding relationship with the identifier of the third pixel.

[0086] In one optional implementation, the first determining module is specifically used for: The identifier of the first pixel is determined based on whether the first pixel in the face region of the current frame face image is located in the face occlusion area of ​​the current frame face image.

[0087] The face image processing apparatus provided in this disclosure first determines the identifier of a first pixel in the face region of the current frame face image, then determines a preset operation corresponding to the identifier of the first pixel, and finally executes the preset operation on the first pixel in the current frame face image. This disclosure, based on an identifier reflecting whether the first pixel has been occluded, implements the execution of the preset operation on the first pixel, enriching the functionality of interactive applications and improving the user experience.

[0088] In addition to the methods and apparatus described above, this disclosure also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the face image processing method described in this disclosure.

[0089] This disclosure also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, implement the face image processing method described in this disclosure.

[0090] In addition, this disclosure also provides a face image processing device, see [link to relevant documentation]. Figure 6 As shown, it may include: The device includes a processor 601, a memory 602, an input device 603, and an output device 604. The number of processors 601 in the face image processing device can be one or more. Figure 6 Taking a processor as an example. In some embodiments of this disclosure, the processor 601, memory 602, input device 603, and output device 604 can be connected via a bus or other means, wherein, Figure 6 Taking the example of a connection between China and Israel via a bus.

[0091] The memory 602 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing of the face image processing device by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The input device 603 can be used to receive input digital or character information, and to generate signal inputs related to user settings and function control of the face image processing device.

[0092] Specifically in this embodiment, the processor 601 will load the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 will run the applications stored in the memory 602 to realize the various functions of the above-mentioned face image processing device.

[0093] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A face image processing method, characterized in that, The method includes: The identifier of the first pixel in the face region of the current frame face image is determined; wherein, the identifier of the first pixel is used to identify whether the face position point corresponding to the first pixel has been occluded, and the face position point has been occluded, including the first pixel being occluded in the current frame face image or a pixel having a positional correspondence with the first pixel being occluded in at least one of the previous n frames face images of the current frame face image, wherein the pixels having a positional correspondence with the first pixel in the previous n frames face images of the current frame face image correspond to the same face position point as the first pixel, and the current frame face image is the face image being displayed during the current shooting process; From the pre-established correspondence between identifiers and preset operations, a preset operation that corresponds to the identifier of the first pixel is determined; the preset operation includes preset effects, and the identifier of the first pixel includes a first preset identifier and a second preset identifier, wherein the identifier of the first pixel is determined to be the first preset identifier or the second preset identifier based on whether the face position point corresponding to the first pixel has been occluded. The preset operation is performed on the first pixel in the current frame face image; The step of performing the preset operation on the first pixel in the current frame face image includes: In response to determining that the identifier of the first pixel in the current frame face image is the first preset identifier, the first preset effect is displayed at the first pixel; or In response to determining that the identifier of the first pixel in the current frame face image is the second preset identifier, the second preset effect is displayed at the first pixel; The step of determining the identifier of the first pixel in the face region of the current frame face image includes: The current frame face image is input into the face occlusion detection model. After processing by the face occlusion detection model, the face occlusion area on the current frame face image is output. The occlusion region of the face in the current frame image is deformed to a standard face to obtain the deformed occlusion region; The identifier of the first pixel is determined based on whether the third pixel on the standard face, which has a positional correspondence with the first pixel on the current frame face image, is located in the deformed face occlusion area and the identifier of the third pixel; wherein, the identifier of the third pixel is updated based on the identifier of the second pixel on the previous frame face image, which has a positional correspondence with the third pixel.

2. The method according to claim 1, characterized in that, After determining the identifier of the first pixel, the method further includes: Based on the identifier of the first pixel, update the identifier of the third pixel on the standard face.

3. The method according to claim 2, characterized in that, The preset operation for determining the correspondence between the identifier of the first pixel includes: A preset operation is determined to have a corresponding relationship with the identifier of the third pixel.

4. A face image processing device, characterized in that, The device includes: The first determining module is used to determine the identifier of a first pixel in the face region of the current frame face image; wherein, the identifier of the first pixel is used to identify whether the face position point corresponding to the first pixel has been occluded, and the face position point being occluded includes the first pixel being occluded in the current frame face image or a pixel having a positional correspondence with the first pixel being occluded in at least one of the previous n frames of the face image, wherein the pixels having a positional correspondence with the first pixel in the previous n frames of the face image correspond to the same face position point as the first pixel, and the current frame face image is the face image being displayed during the current shooting process; The second determining module is used to determine a preset operation that corresponds to the identifier of the first pixel from the pre-established correspondence between identifiers and preset operations; the preset operation includes preset effects, and the identifier of the first pixel includes a first preset identifier and a second preset identifier, wherein the identifier of the first pixel is determined to be the first preset identifier or the second preset identifier based on whether the face position point corresponding to the first pixel has been occluded. The execution module is used to perform the preset operation on the first pixel in the current frame face image; The execution module includes: The first display submodule is configured to, in response to determining that the identifier of the first pixel in the current frame face image is the first preset identifier, display the first preset effect at the first pixel; or The second display submodule is used to display the second preset effect on the first pixel in response to determining that the identifier of the first pixel in the current frame face image is the second preset identifier. The first determining module includes: The second occlusion detection submodule is used to input the current frame face image into the face occlusion detection model, and after processing by the face occlusion detection model, output the face occlusion area on the current frame face image. The deformation submodule is used to deform the face occlusion area of ​​the current frame face image to a standard face, so as to obtain the deformed face occlusion area; The fifth determining submodule is used to determine the identifier of the first pixel based on whether the third pixel on the standard face, which has a positional correspondence with the first pixel on the current frame face image, is located in the deformed face occlusion area and the identifier of the third pixel; wherein, the identifier of the third pixel is updated based on the identifier of the second pixel on the previous frame face image, which has a positional correspondence with the third pixel.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-3.

6. A device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1-3.

Images