Face shape processing method, device, user terminal and computer readable storage medium

Abstract

The present application provides a face shape processing method, device, user terminal and computer-readable storage medium, which relate to the field of image processing. The method is applied to a user terminal, and the method includes: obtaining the current face posture angle information of the target object and the live image corresponding to the current face posture angle information; responding to the user's operation, determining the face shape correction reference; projecting the face point information to the face coordinate system according to the current face posture angle information to obtain the face contour point set of the target object; determining the face shape offset information of the target object according to the face contour point set; the face shape offset information includes the distance between each face contour point in the face contour point set and the X-axis; correcting the face contour according to the face shape correction reference and the face shape offset information to obtain the target image; the face contour is determined by the face point information. Correct the face contour of the target object according to the current face posture angle information of the target object to achieve the beautification of the target object.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a face shape processing method, device, user terminal and computer-readable storage medium. Background Art

[0002] With the development of Internet technology and the improvement of communication functions, images are not only used for display, but also for providing more information to the viewers of the images, and face image processing occupies most of the current image processing scenarios.

[0003] When taking photos for live streaming, the face is naturally asymmetrical, so it does not look natural visually. Currently, face processing only involves face thinning and face shaving, which cannot achieve symmetrical correction of facial contours. Therefore, how to process face images and correct face shape is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present application includes, for example, providing a face shape processing method, device, user terminal and computer-readable storage medium, which can correct the facial contour of the target object according to the current facial posture angle information of the target object to achieve beautification of the target object.

[0005] The embodiments of the present application can be implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a face shape processing method, which is applied to a user terminal, and the method includes:

[0007] Acquire current facial posture angle information of the target object and a live image corresponding to the current facial posture angle information; the live image includes facial point information of the target object;

[0008] Responding to the user's operation, determining the face shape correction benchmark;

[0009] The facial point information is projected onto a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object; the facial coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis;

[0010] Determining face shape offset information of the target object according to the face contour point set; the face shape offset information includes a distance between each face contour point in the face contour point set and the X-axis;

[0011] The facial contour is corrected according to the facial shape correction reference and the facial shape offset information to obtain a target image; the facial contour is determined by the facial point information.

[0012] In an optional embodiment, projecting the facial point information to a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object includes:

[0013] Acquire a coordinate conversion matrix according to the current face posture angle information; the coordinate conversion matrix represents the corresponding relationship between each image coordinate of the live image and the coordinate of the face coordinate system;

[0014] Determining three-dimensional coordinate information of the facial point information in the facial coordinate system according to the coordinate conversion matrix;

[0015] Performing angle conversion on the three-dimensional coordinate information to obtain frontal coordinate information corresponding to the facial point information;

[0016] The edge point information in the front face coordinate information is used as the face contour point set; the face contour point set represents the front face contour of the target object.

[0017] In an optional embodiment, determining the face shape offset information of the target object according to the face contour point set includes:

[0018] Taking the X-axis as the center line, the facial contour point set is divided into a left facial contour point set and a right facial contour point set;

[0019] Subtract each contour point in the left face contour point set from the X axis to obtain a left face offset;

[0020] Subtract each contour point in the right face contour point set from the X axis to obtain a right face offset;

[0021] The face shape offset information is obtained; the face shape offset information includes the left face offset and the right face offset.

[0022] In an optional embodiment, the facial contour of the target object is corrected according to the face shape correction reference and the face shape offset information to obtain a target image, including:

[0023] According to the face shape offset information, the front face coordinate information is shifted toward the face shape correction reference to obtain alignment coordinate information; the alignment coordinate information represents the corrected three-dimensional coordinate information of the front face image of the target object;

[0024] Processing the alignment coordinate information according to the coordinate conversion matrix to obtain two-dimensional coordinate information of the image to be displayed corresponding to the facial point information;

[0025] The two-dimensional coordinate information of the image to be displayed is parsed to obtain a target image corresponding to the live image.

[0026] In an optional embodiment, the interactive interface of the user terminal displays a plurality of face alignment graphics, and the plurality of face alignment graphics correspond one to one to a plurality of face alignment modes;

[0027] The step of responding to the user's operation and determining the face shape correction benchmark includes:

[0028] In response to the user's operation, acquiring a target face shape alignment graphic corresponding to the operation from the multiple face shape alignment graphics;

[0029] The target face shape alignment method corresponding to the target face shape alignment graphic is used as the face shape correction reference.

[0030] In an optional embodiment, the multiple face alignment modes include middle alignment, and the face offset information includes a left face offset and a right face offset;

[0031] The step of correcting the facial contour of the target object according to the facial shape correction reference and the facial shape offset information to obtain a target image includes:

[0032] When the target face alignment mode is the middle alignment, obtaining an average offset of the left face offset and the right face offset;

[0033] obtaining a first image and a second image;

[0034] The first image represents image information obtained by adjusting the left face information of the live image, and the left face offset of the first image is consistent with the average offset; the second image represents image information obtained by adjusting the right face information of the live image, and the right face offset of the second image is consistent with the average offset;

[0035] The first image is fused with the second image to obtain the target image.

[0036] In an optional embodiment, the multiple face shape alignment modes further include first side face alignment; the face shape offset information includes a first side face offset and a second side face offset, the first side face offset represents the distance between each face contour point located on a first side of the X-axis in the face contour point set and the X-axis, and the second side face offset represents the distance between each face contour point located on a second side of the X-axis in the face contour point set and the X-axis;

[0037] The step of correcting the facial contour of the target object according to the facial shape correction reference and the facial shape offset information to obtain a target image includes:

[0038] When the target face alignment method is the first side face alignment, keeping the first side face offset unchanged and adjusting the second side face information in the live image to obtain the target image;

[0039] The second side face information is information of the facial point information located on the second side of the X-axis, and the first side face offset is consistent with the second side face offset of the target image.

[0040] In a second aspect, an embodiment of the present application provides a face shape processing device, applied to a user terminal, the device comprising:

[0041] An acquisition module, used to acquire current facial posture angle information of a target object and a live image corresponding to the current facial posture angle information; the live image includes facial point information of the target object;

[0042] A processing module, used for responding to user operations and determining a face shape correction benchmark;

[0043] A projection module, used to project the facial point information to a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object; the facial coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis;

[0044] A determination module, configured to determine face shape offset information of the target object according to the face contour point set; the face shape offset information includes a distance between each face contour point in the face contour point set and the X-axis;

[0045] The processing module is further used to correct the facial contour according to the facial shape correction reference and the facial shape offset information to obtain a target image; the facial contour is determined by the facial point information.

[0046] In a third aspect, an embodiment of the present application provides a user terminal, comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the method described in any one of the aforementioned embodiments.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the aforementioned embodiments is implemented.

[0048] Compared with the prior art, the present application provides a face shape processing method, device, user terminal and computer-readable storage medium, which relate to the field of image processing. The face shape processing method is applied to a user terminal, and the method comprises: obtaining the current face posture angle information of the target object and the live image corresponding to the current face posture angle information; the live image comprises the face point information of the target object; in response to the user's operation, determining the face shape correction reference; projecting the face point information to the face coordinate system according to the current face posture angle information to obtain the face contour point set of the target object; the face coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line corresponding to the chin point to the nose tip as the X-axis; determining the face shape offset information of the target object according to the face contour point set; the face shape offset information comprises the distance between each face contour point in the face contour point set and the X-axis; correcting the face contour according to the face shape correction reference and the face shape offset information to obtain the target image; the face contour is determined by the face point information. The facial contour of the target object is corrected according to the current facial posture angle information of the target object to achieve beautification of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A block diagram of a user terminal provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a flow chart of a face shape processing method provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of a face image provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a face coordinate system provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of a flow chart of another face shape processing method provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of a flow chart of another face shape processing method provided in an embodiment of the present application;

[0056] Figure 7A schematic diagram of a flow chart of another face shape processing method provided in an embodiment of the present application;

[0057] Figure 8 A display schematic diagram of a user terminal provided in an embodiment of the present application;

[0058] Fig. 9 A flowchart of another face shape processing method is provided for an embodiment of the present application;

[0059] Fig.10 A flowchart of another face shape processing method is provided for an embodiment of the present application;

[0060] Fig.11 A block diagram of a face shape processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiments of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, if the terms "first", "second", etc. appear, they are only used to distinguish the description and cannot be understood as indicating or implying relative importance. It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0063] With the development of Internet technology and the improvement of communication functions, images are not only used for display, but also for providing more information to the viewers of the images, and face image processing occupies most of the current image processing scenarios. When taking photos for live broadcasts, the face has some natural asymmetry, so it does not look natural visually; and the current face processing only has face thinning and face shaving, which cannot achieve symmetrical correction of facial contours. Therefore, how to process face images and correct the face shape is an urgent problem to be solved.

[0064] In order to solve the above problems, the present application embodiment provides a face shape processing method, which is applied to a user terminal. Figure 1 , Figure 1A block diagram of a user terminal provided in an embodiment of the present application, the user terminal 300 may include a processor 310, an internal memory 321, a camera 393, a display screen 394, and a subscriber identification module (Subscriber Identification Module, SIM) card interface, etc.

[0065] The face shape processing method provided in the embodiment of the present application can be applied to terminals such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of user terminal.

[0066] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the user terminal 300. In other embodiments of the present application, the user terminal 300 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0067] The processor 310 may include one or more processing units, for example: the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). A memory may also be provided in the processor 310 for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or circulated. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 310 is reduced, and the efficiency of the system is improved.

[0068] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0069] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the user terminal 300. In other embodiments of the present application, the user terminal 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0070] The wireless communication function of the user terminal 300 can be implemented through an antenna, a mobile communication module, a wireless communication module, a modem processor, and a baseband processor.

[0071] The user terminal 300 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0072] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the user terminal 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.

[0073] The user terminal 300 can realize network live broadcast and video shooting functions through an image signal processor (ISP), a camera 393, a video codec, a GPU, a display screen 394 and an application processor.

[0074] The camera 393 is used to capture static images or dynamic videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the user terminal 300 may include 1 or N cameras 393, where N is a positive integer greater than 1.

[0075] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the user terminal 300 by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the user terminal 300 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0076] The software system of the user terminal 300 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.

[0077] Below Figure 1 Based on the user terminal 300 shown in FIG. 1 , in order to solve the above problems and the deficiencies of the background technology, the embodiment of the present application provides a face shape processing method, such as Figure 2 , Figure 2 A flow chart of a face shape processing method provided in an embodiment of the present application is provided. The face shape processing method is applied to a user terminal 300. The face shape processing method may include the following steps:

[0078] S410, obtaining current facial posture angle information of the target object and a live image corresponding to the current facial posture angle information.

[0079] The live image includes the face point information of the target object. The current face posture angle information can be obtained through the camera 393 of the user terminal 300. The face point information can be obtained by extracting face point information from the image to be processed using face key point extraction methods such as "68 face key points" or "49 face key points". For example, if "68 face key points" are used to extract face point information from the image to be processed, please refer to Figure 3 , Figure 3 A schematic diagram of a facial image provided in an embodiment of the present application, the inner contour information of the facial points of the image to be processed is the facial key points corresponding to numbers 28 to 68, and the outer contour information of the facial points of the image to be processed is the facial key points corresponding to numbers 1 to 27.

[0080] S420, responding to the user's operation and determining a face shape correction benchmark.

[0081] For example, if the user terminal 300 is a mobile terminal with a touch screen, such as a tablet computer or a mobile phone, the operation may be input by the user through an input device connected to the mobile terminal, such as a mouse, a keyboard, etc., or may be generated by the user clicking the touch screen, or may be generated by the user sliding a preset gesture operation on the touch screen by a knuckle (e.g., a knuckle gesture operation in an "L" shape on the touch screen). The touch screen may be connected to the above Figure 1 The display screen 394 shown is integrated, or may be separately provided from the display screen 394. In another possible case, the mobile terminal may have a sensor for identifying user gestures in the air, and the user may perform gesture operations in the air (i.e., the user is not in direct contact with the mobile terminal) so that the mobile terminal determines the face shape correction benchmark corresponding to the operation.

[0082] S430, projecting the facial point information to the facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object.

[0083] The face coordinate system is a three-dimensional coordinate system with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis. The facial contour point set represents the frontal face contour of the target object. Figure 4 , Figure 4 A schematic diagram of a facial coordinate system provided in an embodiment of the present application, wherein a three-dimensional coordinate system is established with the chin point of the target object as the origin and the straight line corresponding to the chin point to the nose tip as the X-axis, the Y-axis of the facial coordinate system can be parallel to the horizon, the XY plane of the facial coordinate system is parallel to the frontal image of the target image, and the Z-axis of the facial coordinate system is a straight line perpendicular to the frontal image of the target image.

[0084] S440: Determine face shape offset information of the target object according to the face contour point set.

[0085] The face shape offset information includes the distance between each face contour point in the face contour point set and the X axis. For example, please continue to refer to Figure 4 , Figure 4 In FIG. 1 , some facial contour points of the target object are highlighted to show the relationship between the facial contour points and the X-axis. The target object also has other facial contour points.

[0086] S450, correcting the facial contour according to the face shape correction reference and the face shape offset information to obtain a target image.

[0087] The facial contour is determined by facial point information; for example, edge point information of the facial point information is used as the facial contour of the target object in the live image.

[0088] For example, with respect to the above-mentioned face shape correction benchmark, it may include, but is not limited to, center alignment, left face alignment, and right face alignment; left face alignment is to project the left and right faces of the target object onto the X-axis, calculate the offset of the left and right face projections, and move the right face to align with the left face based on the left face; right face alignment is to project the left and right faces of the target object onto the X-axis, calculate the offset of the left and right face projections, and move the left face to align with the right face based on the right face; center alignment is to project the left and right faces of the target object onto the X-axis, calculate the middle value of the left and right face projections, and use it as a reference to move both the left and right faces to the middle value for alignment.

[0089] It should be understood that using the face shape processing method provided in the embodiment of the present application, the face point information of the target object can be processed according to the current face posture angle information of the target object, the face contour of the target object can be corrected, the face shape of the target object can be corrected, and the beauty effect of the live broadcast application (Application, APP) can be improved.

[0090] If the facial contour of the target object is obtained only based on the facial point information in the live image, which is two-dimensional information, when the target object moves, if the face in the live image is a side face, it is difficult to accurately correct the face of the target object. In an optional embodiment, in order to solve the above problem, Figure 2 Based on this, a possible implementation is given, see Figure 5 , Figure 5 A flow chart of another face shape processing method provided in an embodiment of the present application, for the above S430: projecting the face point information to the face coordinate system according to the current face posture angle information to obtain the face contour point set of the target object, which may include:

[0091] S4301, obtaining a coordinate transformation matrix according to the current face posture angle information.

[0092] The coordinate transformation matrix represents the correspondence between each image coordinate of the live image and the coordinate of the face coordinate system.

[0093] S4302, determining the three-dimensional coordinate information of the facial point information in the facial coordinate system according to the coordinate conversion matrix.

[0094] For example, the coordinate rotation matrix can be used to transform each image coordinate (two-dimensional coordinate information) of the live image with the coordinate (three-dimensional coordinate information) of the face coordinate system to improve the accuracy of face shape correction.

[0095] S4303, performing angle conversion on the three-dimensional coordinate information to obtain the frontal face coordinate information corresponding to the facial point information.

[0096] It should be understood that the above three-dimensional coordinate information can be the target object relative to the image acquisition unit of the user terminal (such as Figure 1 The camera 393 shown in the figure is in a side face state, and the three-dimensional coordinate information is converted into an angle to obtain the target object relative to the image acquisition unit (such as Figure 1 The frontal coordinate information of the camera 393) shown in the figure when it is in the frontal state can improve the accuracy of face shape correction.

[0097] S4304: Use the edge point information in the front face coordinate information as a facial contour point set.

[0098] The facial contour point set represents the frontal facial contour of the target object. In other words, edge points are obtained for the area surrounded by the frontal facial coordinate information, and the edge point information is used as the facial contour point set. The facial contour of the target object can be obtained according to the facial contour point set, thereby realizing the correction of the facial contour in the live image.

[0099] In an optional embodiment, in order to obtain face shape offset information, Figure 5 Based on this, a possible implementation is given, see Figure 6 , Figure 6 A flowchart of another face shape processing method provided in an embodiment of the present application is provided, with respect to the above-mentioned S440: determining face shape offset information of a target object according to a face contour point set, which may include:

[0100] S4401, taking the X axis as the center line, dividing the facial contour point set into a left facial contour point set and a right facial contour point set.

[0101] Please continue to see Figure 4 , taking the X-axis of the face coordinate system as the center line, multiple facial contour points on the left are divided into a left face contour point set, and multiple facial contour points on the right are divided into a right face contour point set.

[0102] S4402, subtract each contour point in the left face contour point set from the X axis to obtain a left face offset.

[0103] Please continue to see Figure 4 The four horizontal lines on the left represent the left face offset obtained by subtracting each contour point in the left face contour point set from the X axis.

[0104] S4403, subtract each contour point in the right face contour point set from the X axis to obtain a right face offset.

[0105] Please continue to see Figure 4 The four horizontal lines on the right represent the right face offset obtained by subtracting each contour point in the right face contour point set from the X-axis.

[0106] S4404, obtain face shape offset information.

[0107] The face shape offset information includes a left face offset and a right face offset. In other words, the face offset information includes the distance between each face contour point in the face contour point set and the X axis.

[0108] It can be understood that the facial offset information is represented in the face coordinate system, and the facial offset information is used to correct the facial contour of the target object, and the coordinate transformation is performed according to the coordinate rotation matrix corresponding to the current face posture angle information, so that the target image face correction effect is more natural.

[0109] In an optional embodiment, in order to make the face shape correction effect of the target image more natural, Figure 5 Based on this, a possible implementation is given, see Figure 7 , Figure 7 A flowchart of another face shape processing method provided in an embodiment of the present application, with respect to the above-mentioned S450: correcting the face contour of the target object according to the face shape correction reference and the face shape offset information to obtain a target image, may include:

[0110] S4501, according to the face shape offset information, the front face coordinate information is shifted toward the face shape correction reference to obtain the alignment coordinate information.

[0111] The alignment coordinate information represents the three-dimensional coordinate information of the corrected frontal face image of the target object. For example, if the face shape correction reference is the left face, that is, the face shape correction alignment is performed according to the left face of the target object, then the right face is moved to align with the left face based on the left face of the target object.

[0112] S4502, processing the alignment coordinate information according to the coordinate conversion matrix to obtain the two-dimensional coordinate information of the image to be displayed corresponding to the facial point information.

[0113] It should be understood that when the coordinate conversion matrix is ​​used to process the alignment coordinate information, what is actually used is the inverse matrix corresponding to the coordinate conversion matrix, so as to realize the conversion of the three-dimensional coordinate information into the two-dimensional coordinate information.

[0114] S4503, parsing the two-dimensional coordinate information of the image to be displayed to obtain a target image corresponding to the live image.

[0115] It should be understood that the two-dimensional coordinate information of the image to be displayed is the pixel coordinate information of the target object, and the RGB value corresponding to each pixel coordinate can be obtained according to the live image, and then the target image corresponding to the live image is obtained to realize the facial contour correction of the target object.

[0116] In an optional embodiment, the face shape correction benchmark may be a default one, for example, the default face shape correction benchmark is middle alignment, that is, the left and right face offsets are aligned to the average offset of the two to perform face shape correction. In order to enrich the face shape correction method and improve the symmetry effect of face shape correction, Figure 1 Based on the example, the user terminal 300 is a mobile phone. Figure 8 , Figure 8 A display diagram of a user terminal provided in an embodiment of the present application is shown, wherein the interactive interface of the user terminal 300 displays multiple face alignment graphics, and the multiple face alignment graphics correspond one to one with multiple face alignment methods. The multiple face alignment methods may include, but are not limited to: left face alignment, right face alignment, and middle alignment.

[0117] Below Figure 8 Taking the user terminal 300 shown in the figure as an example, in order to enrich the face shape correction method and improve the symmetry effect of face shape correction, Figure 2 Based on this, a possible implementation is given, see Fig. 9 , Fig. 9 A flowchart of another face shape processing method is provided for an embodiment of the present application. With respect to the above-mentioned S420: responding to the user's operation, determining the face shape correction benchmark, which may include:

[0118] S4201, responding to a user operation, obtaining a target face alignment graphic corresponding to the operation from a plurality of face alignment graphics.

[0119] The operation may be generated by the user clicking the touch screen, or by the user sliding a preset gesture operation with a knuckle on the touch screen (e.g., a knuckle following an "L"-shaped gesture operation on the touch screen). The touch screen may be connected to the above Figure 1 The display screen 394 shown is integrated, or may be provided separately from the display screen 394.

[0120] S4202, using the target face shape alignment method corresponding to the target face shape alignment graphic as a face shape correction reference.

[0121] For example, see Figure 8 ,trigger Figure 8 If you click the face alignment graphic marked as "Face Alignment" in the image, you can set the face correction benchmark to center alignment. The face alignment graphic can be triggered by clicking on the touch screen or by gesture operation in the air.

[0122] That is to say, the target face shape alignment graphic is determined through operation, and then the target face shape alignment method corresponding to the target face shape alignment graphic is used as the face shape correction reference, and the face point information of the target object is processed according to the current face posture angle information, and the face contour of the target object is corrected to achieve face shape correction of the target object and improve the beauty effect of the live broadcast APP.

[0123] In the process of symmetric correction of the target object's facial contour, Fig. 9 Based on this, a possible implementation method is given by taking multiple face alignment methods including middle alignment and face offset information including left face offset and right face offset as an example. Fig.10 , Fig.10 A flow chart of another face shape processing method is provided for an embodiment of the present application, with respect to S450: correcting the face contour of the target object according to the face shape correction reference and the face shape offset information to obtain a target image, including:

[0124] S450a: When the target face alignment mode is middle alignment, obtain an average offset of the left face offset and the right face offset.

[0125] S450b, obtaining a first image and a second image.

[0126] Among them, the first image represents the image information obtained by adjusting the left face information of the live image, and the left face offset of the first image is consistent with the average offset; the second image represents the image information obtained by adjusting the right face information of the live image, and the right face offset of the second image is consistent with the average offset.

[0127] S450c, fusing the first image with the second image to obtain a target image.

[0128] For example, the left face part in the first image is fused with the right face part in the second image to obtain a target image, thereby achieving symmetric correction of the left and right faces in the target image.

[0129] In an optional embodiment, the multiple face shape alignment methods further include first side face alignment. The face shape offset information includes a first side face offset and a second side face offset, the first side face offset represents the distance between each face contour point located on the first side of the X-axis and the X-axis in the face contour point set, and the second side face offset represents the distance between each face contour point located on the second side of the X-axis and the X-axis in the face contour point set.

[0130] For example, the first side face alignment can be a left face alignment or a right face alignment: if the first side face alignment is a left face alignment, the first side face offset is the left face offset, and the second side face offset is the right face offset; if the first side face alignment is a right face alignment, the first side face offset is the right face offset, and the second side face offset is the left face offset.

[0131] Regarding the above S450: correcting the facial contour of the target object according to the face shape correction reference and the face shape offset information to obtain the target image, it may include: when the target face shape alignment method is the first side face alignment, keeping the first side face offset unchanged and adjusting the second side face information in the live image to obtain the target image. The second side face information is the information located on the second side of the X-axis in the face point information, and the first side face offset is consistent with the second side face offset of the target image.

[0132] That is to say, when the facial contour of the target object is symmetrically corrected, the facial information on one side can be used as a face shape correction reference, and the facial information on the other side can be aligned with the facial information on the one side to complete the facial contour correction of the target.

[0133] In order to implement the face shape processing method provided in any of the above embodiments, the present application embodiment provides a face shape processing device applied to a user terminal, see Fig.11 , Fig.11 A block diagram of a face shape processing device provided in an embodiment of the present application, the face shape processing device 50 comprises: an acquisition module 51, a processing module 52, a projection module 53 and a determination module 54.

[0134] The acquisition module 51 is used to acquire the current face posture angle information of the target object and the live image corresponding to the current face posture angle information. The live image includes the face point information of the target object.

[0135] The processing module 52 is used to respond to the user's operation and determine the face shape correction benchmark.

[0136] The projection module 53 is used to project the facial point information to the facial coordinate system according to the current facial posture angle information to obtain the facial contour point set of the target object. The facial coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis.

[0137] The determination module 54 is used to determine the face shape offset information of the target object according to the face contour point set. The face shape offset information includes the distance between each face contour point in the face contour point set and the X axis.

[0138] The processing module 52 is also used to correct the facial contour according to the facial shape correction reference and the facial shape offset information to obtain a target image. The facial contour is determined by the facial point information.

[0139] The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a face shape processing method as described in any of the above embodiments is implemented. The computer-readable storage medium can be, but is not limited to, a U disk, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic disk, or an optical disk, and other media that can store program codes.

[0140] In summary, the present application provides a face shape processing method, device, user terminal and computer-readable storage medium, which relate to the field of image processing. The face shape processing method is applied to a user terminal, and the method includes: obtaining the current face posture angle information of the target object and the live image corresponding to the current face posture angle information; the live image includes the face point information of the target object; responding to the user's operation, determining the face shape correction reference; projecting the face point information to the face coordinate system according to the current face posture angle information, and obtaining the face contour point set of the target object; the face coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line corresponding to the chin point to the nose tip as the X-axis; determining the face shape offset information of the target object according to the face contour point set; the face shape offset information includes the distance between each face contour point in the face contour point set and the X-axis; correcting the face contour according to the face shape correction reference and the face shape offset information to obtain the target image; the face contour is determined by the face point information. Correct the face contour of the target object according to the current face posture angle information of the target object to achieve the beauty of the target object.

[0141] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A face shape processing method, characterized in that: Applied to a user terminal, the interactive interface of the user terminal displays a plurality of face alignment graphics, the plurality of face alignment graphics correspond one to one to a plurality of face alignment modes, the plurality of face alignment modes including middle alignment; The method comprises: Acquire current facial posture angle information of the target object and a live image corresponding to the current facial posture angle information; the live image includes facial point information of the target object, and the facial point information is used to determine the facial contour; In response to the user's operation, acquiring a target face shape alignment graphic corresponding to the operation from the multiple face shape alignment graphics; Using the target face shape alignment method corresponding to the target face shape alignment graphic as a face shape correction reference; The facial point information is projected onto a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object; the facial coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis; Determining face shape offset information of the target object according to the face contour point set; the face shape offset information includes a distance between each face contour point in the face contour point set and the X-axis, a left face offset, and a right face offset; Correcting the facial contour of the target object according to the face shape correction reference and the face shape offset information to obtain a target image, including: When the target face alignment mode is the middle alignment, obtaining an average offset of the left face offset and the right face offset; obtaining a first image and a second image; The first image represents image information obtained by adjusting the left face information of the live image, and the left face offset of the first image is consistent with the average offset; the second image represents image information obtained by adjusting the right face information of the live image, and the right face offset of the second image is consistent with the average offset; The first image is fused with the second image to obtain a target image.

2. The method according to claim 1, characterized in that: The step of projecting the facial point information onto a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object comprises: Acquire a coordinate conversion matrix according to the current face posture angle information; the coordinate conversion matrix represents the corresponding relationship between each image coordinate of the live image and the coordinate of the face coordinate system; Determining three-dimensional coordinate information of the facial point information in the facial coordinate system according to the coordinate conversion matrix; Performing angle conversion on the three-dimensional coordinate information to obtain frontal coordinate information corresponding to the facial point information; The edge point information in the front face coordinate information is used as the face contour point set; the face contour point set represents the front face contour of the target object.

3. The method according to claim 2, characterized in that Determining face shape offset information of the target object according to the face contour point set includes: Taking the X-axis as the center line, the facial contour point set is divided into a left facial contour point set and a right facial contour point set; Subtract each contour point in the left face contour point set from the X axis to obtain a left face offset; Subtract each contour point in the right face contour point set from the X axis to obtain a right face offset; The face shape offset information is obtained; the face shape offset information includes the left face offset and the right face offset.

4. The method according to claim 2, characterized in that: The step of correcting the facial contour of the target object according to the facial shape correction reference and the facial shape offset information to obtain a target image includes: According to the face shape offset information, the front face coordinate information is shifted toward the face shape correction reference to obtain alignment coordinate information; the alignment coordinate information represents the corrected three-dimensional coordinate information of the front face image of the target object; Processing the alignment coordinate information according to the coordinate conversion matrix to obtain two-dimensional coordinate information of the image to be displayed corresponding to the facial point information; The two-dimensional coordinate information of the image to be displayed is parsed to obtain a target image corresponding to the live image.

5. The method according to claim 1, characterized in that: The multiple face shape alignment modes further include first side face alignment; the face shape offset information includes a first side face offset and a second side face offset, the first side face offset represents the distance between each face contour point located on a first side of the X-axis in the face contour point set and the X-axis, and the second side face offset represents the distance between each face contour point located on a second side of the X-axis in the face contour point set and the X-axis; The step of correcting the facial contour of the target object according to the facial shape correction reference and the facial shape offset information to obtain a target image includes: When the target face alignment method is the first side face alignment, keeping the first side face offset unchanged and adjusting the second side face information in the live image to obtain the target image; The second side face information is information of the facial point information located on the second side of the X-axis, and the first side face offset is consistent with the second side face offset of the target image.

6. A face shape processing device, characterized in that: Applied to a user terminal, the interactive interface of the user terminal displays a plurality of face alignment graphics, the plurality of face alignment graphics correspond one to one to a plurality of face alignment modes, the plurality of face alignment modes including middle alignment; The device comprises: An acquisition module, used to acquire current facial posture angle information of a target object and a live image corresponding to the current facial posture angle information; the live image includes facial point information of the target object, and the facial point information is used to determine a facial contour; The processing module is used to: in response to the user's operation, obtain a target face shape alignment figure corresponding to the operation from the multiple face shape alignment figures; and use the target face shape alignment method corresponding to the target face shape alignment figure as a face shape correction reference; A projection module, used to project the facial point information to a facial coordinate system according to the current facial posture angle information to obtain a facial contour point set of the target object; the facial coordinate system is a three-dimensional coordinate system established with the chin point of the target object as the origin and the straight line from the chin point to the nose tip as the X-axis; A determination module, configured to determine face shape offset information of the target object according to the face contour point set; the face shape offset information includes a distance between each face contour point in the face contour point set and the X-axis, a left face offset, and a right face offset; The processing module is also used to: when the target face alignment method is the middle alignment, obtain the average offset of the left face offset and the right face offset; obtain a first image and a second image; wherein the first image represents image information obtained by adjusting the left face information of the live image, and the left face offset of the first image is consistent with the average offset; the second image represents image information obtained by adjusting the right face information of the live image, and the right face offset of the second image is consistent with the average offset; and fuse the first image and the second image to obtain a target image.

7. A user terminal, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images