Image processing method and device and storage medium

Abstract

The invention provides an image processing method and device and a storage medium, and is applied to the technical field of terminals. The method comprises the steps that under the condition that the electronic equipment starts a facial beautification function of a camera application, the electronic equipment obtains an original image which is collected by a camera and comprises a portrait face, after image processing, the image after facial beautification processing is displayed on a shooting interface of the camera application, and the image after facial beautification processing is an image obtained after the original image is superposed with a cosmetic mask image; the cosmetic mask pattern is a mask pattern matched with the face shape of the face of the portrait, and the cosmetic mask pattern is used for indicating the positions, shapes and degrees of a highlight area and a shadow area of the face of the portrait. According to the method, the face shape difference of different portraits is considered, and the cosmetic mask image matched with the face shape of the face of the portraits is superposed on the original image, so that the personalized beauty requirements of different users are met, and the portrait beauty effect is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly to an image processing method, device, and storage medium. Background Art

[0002] With the popularization of intelligent electronic devices, users use electronic devices such as mobile phones and tablets to take pictures. Some users will use the beauty function to optimize the face shape, adjust the skin color, set makeup, etc.

[0003] At present, the beauty function is usually presented to users in the form of beauty templates. Users can select their favorite beauty templates from the beauty template library and then take a picture with a beauty effect.

[0004] Since the parameters in the beauty template, such as various deformation parameters of the face, makeup parameters, etc., are a set of fixed configuration parameters, it is impossible to meet the different beauty needs of different users. Summary of the Invention

[0005] Embodiments of this application provide an image processing method, device, and storage medium, which can meet the personalized beauty needs of different users and improve the beauty effect of portraits.

[0006] In a first aspect, an embodiment of this application proposes an image processing method applied to an electronic device. The method includes: in response to an operation of turning on the beauty function of the camera application, obtaining a first image including a human face captured by the camera; displaying a third image on the shooting interface of the camera application, where the third image is an image obtained by superimposing a contour masking image on the first image, and the contour masking image is a masking image matching the face shape of the human face, and the contour masking image is used to indicate the positions, shapes, and degrees of the highlight area and the shadow area of the human face. The first image is the original image captured by the camera. The above method considers the differences in different human face shapes, and superimposes a contour masking image matching the face shape of the human face on the original image to meet the personalized beauty needs of different users and improve the beauty effect of portraits.

[0007] In an optional embodiment of the first aspect, before displaying the third image on the shooting interface of the camera application, the method further includes: obtaining the feature information of the human face in the first image; determining the face shape of the human face based on the feature information of the human face; obtaining a contour masking image matching the face shape of the human face from the database, where the database includes contour masking images corresponding to different face shapes; performing a superimposing process on the first image and the contour masking image to obtain the third image. The above method determines the human face shape by obtaining the human face features, and obtains the contour masking image corresponding to the human face shape to improve the contour of the human face in the image.

[0008] In an optional embodiment of the first aspect, obtaining the feature information of the portrait face in the first image includes: determining the image area where the portrait face is located in the first image; and identifying the key points of each part of the portrait face in the first image through a preset facial key point detection model to obtain the key point information of the portrait face in the first image. The key point information includes the position information of the key points of each part of the portrait face. The above method provides data support for face shape analysis, matching, and facial adjustment by detecting through the facial key point detection model to obtain the key point information of each part of the portrait face.

[0009] In an optional embodiment of the first aspect, determining the face shape of the portrait face based on the feature information of the portrait face includes: determining the facial contour information of the portrait based on the key point information of the portrait face in the first image; determining the face shape of the portrait face based on the facial contour information; or, determining the face shape similarity and five - sense organ similarity between the portrait face in the first image and multiple standard faces through a preset portrait similarity model, determining the total similarity between the portrait face and multiple standard faces, and taking the face shape of the standard face with the maximum total similarity as the face shape of the portrait face in the first image. The first method of determining the portrait face shape above determines the facial contour through portrait key points and matches the corresponding face shape based on the facial contour. The second method of determining the portrait face shape above is based on the portrait face similarity model, calculates the similarity between the portrait face shape and different standard face shapes, and then determines the portrait face shape.

[0010] In an optional embodiment of the first aspect, the positions of the high - light area and the shadow area of the portrait face in the contour mask map corresponding to different face shapes are different, and the shapes of the high - light area and the shadow area of the portrait face in the contour mask map corresponding to different face shapes are different.

[0011] In an optional embodiment of the first aspect, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map corresponding to different face shapes are different, including: if the face shape of the portrait face is an oval face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of a water droplet; or, if the face shape of the portrait face is a round face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of a lightning bolt; or, if the face shape of the portrait face is a rectangular face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of a comb; or, if the face shape of the portrait face is a diamond face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of a semi - arc; or, if the face shape of the portrait face is an elliptical face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of a Z; or, if the face shape of the portrait face is a square face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask map are in the shape of an L.

[0012] The above two embodiments illustrate the differences in the contour masks corresponding to different face shapes. After determining the face shape of a portrait, by matching the contour mask corresponding to the face shape of the portrait, differential contouring of the portrait face can be achieved, enhancing the beauty effect of the portrait.

[0013] In an alternative embodiment of the first aspect, the method further includes: determining the deformation parameters of each part of the face based on the feature information of the portrait face; adjusting the structure of the portrait face in the first image region by region based on the deformation parameters of each part of the face to obtain a second image; and performing an overlay process on the first image and the contour mask, including: performing an overlay process on the second image and the contour mask to obtain a third image. The above method first optimizes the structure of the portrait face, and on the basis of the optimized face structure, performs contouring on the portrait face to enhance the beauty effect of the portrait. It should be noted that when optimizing the structure of the portrait face, the structure of each part of the portrait face is adjusted region by region to avoid affecting adjacent parts.

[0014] In an alternative embodiment of the first aspect, determining the deformation parameters of each part of the face based on the feature information of the portrait face includes: adaptively adjusting the parameters of the structure of the portrait face in the first image within a preset numerical range based on the feature information of the portrait face and the facial feature information of a standard face, so as to determine the deformation parameters of each part of the portrait face. The deformation parameters of each part of the face should be set within a reasonable numerical range, that is, the deformation parameters of each part should be set within the preset numerical range corresponding to each part. It can be understood that if the deformation parameter is too large, it may cause facial distortion, and if the deformation parameter is too small, the optimization of the facial structure is not obvious. By means of adaptive parameter adjustment, reasonable facial deformation parameters are generated to optimize the structure of the portrait face.

[0015] In an alternative embodiment of the first aspect, the standard face includes standard faces of multiple different face shapes; determining the deformation parameters of each part of the face based on the feature information of the portrait face and the facial feature information of the standard face, including: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; and adaptively adjusting the parameters of the structure of the portrait face in the first image within a preset numerical range based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face, so as to determine the deformation parameters of each part of the portrait face. The above method combines the facial features of the standard face corresponding to the portrait face shape to adaptively adjust the parameters of each part of the portrait face, so as to generate reasonable facial deformation parameters and optimize the structure of the portrait face.

[0016] In an alternative embodiment of the first aspect, the method further includes: in response to a first operation of turning on the portrait shooting mode of the camera application, turning on the beauty effect function. Alternatively, in response to the first operation of turning on the portrait shooting mode, a first control is displayed on the shooting interface, and the first control is used to trigger the turning on or off of the beauty effect function; in response to a second operation on the first control, the beauty effect function is turned on.

[0017] Exemplarily, referring to Figure 1 , the first operation may be an operation in which the user selects the portrait shooting mode in the shooting mode selection area 105 of the shooting interface 101 of the camera application. In one example, in response to the first operation, the electronic device directly turns on the beauty effect function. In another example, in response to the first operation, the camera application switches to the portrait shooting mode, and at this time, the beauty effect function is not turned on. Continuing to refer to Figure 1 , the first control may be the beauty effect function switch 103 in the preview area 102 of the shooting interface 101 of the camera application, and the second operation may be an operation of clicking the beauty effect function switch 103. In response to the second operation, the electronic device turns on the beauty effect function. Different from the previous example, the beauty effect function needs to be turned on by the user independently.

[0018] The above method shows two methods of turning on the beauty effect function. After the beauty effect function is turned on, the electronic device can execute the above image processing method to meet the personalized beauty effect needs of different users.

[0019] In a second aspect, an embodiment of the present application provides an image processing apparatus, including: an acquisition module, configured to acquire a first image including a human portrait face collected by a camera in response to an operation of turning on the beauty effect function of the camera application; a display module, configured to display a third image on the shooting interface of the camera application, where the third image is an image after the first image is superimposed with a contour masking map, and the contour masking map is a masking map matching the face shape of the human portrait face, and the contour masking map is used to indicate the positions, shapes, and degrees of the highlight area and the shadow area of the human portrait face.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor, where the processor is configured to call a computer program in the memory to execute the method according to any one of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a chip, including a processor, where the processor is configured to call a computer program in the memory to execute the method according to any one of the first aspect.

[0022] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program runs on an electronic device, the electronic device is caused to execute the method according to any one of the first aspect.

[0023] Sixth aspect, a computer program product includes a computer program which, when run, causes a computer to execute the method according to any one of the first aspect.

[0024] It should be understood that the technical solutions of the second to sixth aspects of the present application correspond to those of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding optional embodiments are similar and will not be elaborated herein. Description of the Drawings

[0025] Figure 1 Schematic diagram of the interface change of the electronic device provided in the embodiment of the present application;

[0026] Figure 2 General flowchart of the image processing method provided in the embodiment of the present application;

[0027] Figure 3 Schematic diagram of the process of the image processing method provided in the embodiment of the present application Figure 1 ;

[0028] Figure 4 Schematic diagram of the facial key points provided in the embodiment of the present application;

[0029] Figure 5 Schematic diagram of setting the deformation parameters of the portrait facial structure provided in the embodiment of the present application;

[0030] Figure 6 Schematic diagram of the process of the image processing method provided in the embodiment of the present application Figure 2 ;

[0031] Figure 7 Schematic diagram of the contour masks corresponding to different face shapes provided in the embodiment of the present application;

[0032] Figure 8a Schematic diagram of the contouring method corresponding to the oval face provided in the embodiment of the present application;

[0033] Figure 8b Schematic diagram of the contouring method corresponding to the round face provided in the embodiment of the present application;

[0034] Figure 8c Schematic diagram of the contouring method corresponding to the rectangular face provided in the embodiment of the present application;

[0035] Figure 8d Schematic diagram of the contouring method corresponding to the diamond face provided in the embodiment of the present application;

[0036] Figure 8e Schematic diagram of the contouring method corresponding to the elliptical face provided in the embodiment of the present application;

[0037] Figure 8fSchematic diagram of the contouring method corresponding to a square face provided by an embodiment of the present application;

[0038] Figure 9 Schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0039] Figure 10 Schematic software architecture diagram of the electronic device provided by an embodiment of the present application;

[0040] Figure 11 Schematic flowchart of the image processing method provided by an embodiment of the present application Figure 3 。 Detailed implementation manners

[0041] For the convenience of clearly describing the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily mean different.

[0042] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0043] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (kind / individual) of the following" or its similar expressions refer to any combination of these items, including any combination of single item (kind / individual) or plural items (kinds / individuals). For example, at least one (kind / individual) of a, b or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple.

[0044] It should be noted that the user information (including but not limited to user device information, user personal information, user facial information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0045] When the user uses the application to take pictures or videos, the application provides a beauty function for the user. The existing beauty functions include filters in different styles, and different degrees of skin smoothing, whitening, makeup effect, face slimming, eye enlarging, nose slimming, head shrinking, etc. After the user opens the application, for example, taking a short video application as an example, when the user wants to perform beauty when shooting a short video, the user can click on different beauty templates provided below the screen to optimize the user's facial features and overall shooting style.

[0046] However, since the parameters in the beauty template are usually fixed configuration parameters, it is impossible to achieve a thousand-person-thousand-variation for the different beauty needs of different users. For example, if a user has relatively large eyes and the eye deformation parameter in the beauty template is relatively large, after using this beauty template, the user will have a serious problem of eye distortion. Another example is that if a user has a round face and the contouring area of the chin in the beauty template is relatively small, after using this beauty template, the face slimming effect is not good.

[0047] In view of the above problems, the embodiment of this application shows an image processing method. When the beauty function is started, the electronic device obtains the facial feature information of the user in the image collected by the camera, determines the deformation parameters of each part of the user's face based on the facial feature information, and realizes precise facial structure optimization. In addition, the electronic device can also determine the user's face shape based on the facial feature information. After completing the structural optimization of the user's face, it can also match the contouring parameters corresponding to the face shape based on the user's face shape to improve the beauty effect. The above method performs personalized beauty based on the facial features of different users, and different users use different beauty parameters, which will not cause beauty distortion.

[0048] In the embodiment of this application, the beauty parameters include deformation parameters of each part of the user's face, makeup parameters, etc. The deformation parameters of each part of the user's face include but are not limited to face slimming parameters, eye enlarging parameters, nose slimming parameters, head shrinking parameters, etc. The makeup parameters include contouring parameters, and the contouring parameters include the position, shape, and degree of facial highlights and shadows.

[0049] In some embodiments, the beauty parameters further include skin smoothing parameters, whitening parameters, and skin tone parameters.

[0050] In some embodiments, the makeup parameters also include lip parameters, blush parameters, eyebrow dyeing parameters, eye bags parameters, eye light parameters, eye highlight parameters, eye shadow parameters, eyelash parameters, etc.

[0051] The user can manually turn on the switch of the beauty function in the shooting interface of the camera application to activate the beauty function. Alternatively, after the user selects the portrait shooting mode in the shooting interface of the camera application, the camera application automatically turns on the beauty function. The embodiment of the present application does not limit the method of turning on the beauty function.

[0052] The portrait shooting mode can determine the background based on the portrait in the image and blur the background to make the background appear blurred, that is, background blur.

[0053] For example, Figure 1 The following is a schematic diagram of the interface changes of the electronic device provided in the embodiment of the present application, taking the electronic device as a mobile phone as an example, such as Figure 1 As shown in a in FIG. 1 , the user opens the camera application in the mobile phone and selects the portrait shooting mode in the shooting mode selection area 104. The preview area 102 of the shooting interface 101 can display the switch 103 of the beauty function. At this time, the portrait in the preview area 102 is the original portrait taken by the camera. The user clicks the switch 103 to turn on the beauty function. When the mobile phone detects that there is a portrait face in the image captured by the camera, the image processing method provided in the embodiment of the present application can be executed to realize personalized beauty of the portrait face in the image, such as Figure 1 As shown in b in FIG. 1 , the portrait in the preview area 105 is a portrait after beauty processing, such as making the nose thinner, lips thicker, and face contouring.

[0054] Understandably, Figure 1 The interface shown merely illustrates a possible interface style of an electronic device taking a mobile phone as an example and should not constitute a limitation on the embodiments of the present application.

[0055] Combination Figure 1 Example, Figure 2 The overall process of the image processing method provided in the embodiment of the present application is shown. When the beauty function is activated in the electronic device, Figure 2 As shown, the image processing process includes the following two processes:

[0056] First processing step: the electronic device performs facial recognition on the image captured by the camera to obtain the facial feature information of the user in the image, and based on the facial feature information, determines the deformation parameters of each part of the face to adjust the facial structure of the user. For example, Figure 2In the user, the width of the nasolabial base is relatively wide, and the thickness of the upper lip is relatively thin. In the first processing procedure, fine-tuning is mainly performed on the nasolabial base and the upper lip of this user. It should be understood that the facial features of different users are different, and the local parts of the user's face can be adaptively adjusted accordingly to avoid distortion of the facial structure to meet the beauty needs of different users.

[0057] Second processing procedure: The electronic device determines the user's face shape based on the facial feature information, matches corresponding contouring parameters based on the user's face shape, and performs contouring processing on the user's face based on the contouring parameters to obtain the image after beauty processing. For example, Figure 2 As shown in d in, performing contouring processing on the user's face is mainly to add shadows and highlights to the user's face. It should be understood that the face shapes of different users are different, and contouring processing can be performed on the user's face accordingly to enhance the contouring effect of the user's face.

[0058] The first processing procedure and the second processing procedure will be described in detail below.

[0059] Exemplarily, Figure 3 is a flowchart illustration of the image processing method provided by an embodiment of this application Figure 1 . Figure 3 The image processing method shown mainly relates to the above-mentioned first processing procedure. The method includes:

[0060] S301. Obtain a first image collected by a camera.

[0061] After the camera application is turned on, the camera of the electronic device can collect images in real time. The images collected by the camera are also called original images. At a certain moment, the image collected by the camera is the first image.

[0062] S302. Perform face recognition on the first image to determine whether a human face is included in the first image.

[0063] After receiving the first image collected by the camera, the electronic device can perform face recognition on the first image through a pre-set face recognition model to determine whether a human face is included in the first image. If a human face is included in the first image, confirm the image area where the human face is located and execute S303.

[0064] S303. Obtain the facial feature information of the human face in the first image.

[0065] After determining the image area where the human face in the first image is located, the electronic device can identify the key points of each part of the human face in the first image through a pre-set facial key point detection model to obtain the facial feature information of the human face in the first image. The facial feature information includes the key point information of the human face.

[0066] The input of the facial key point detection model can be the image area where the portrait face determined by the face recognition model is located. Since the background area in the first image is removed from the model input, the processing speed of the facial key point detection model can be improved.

[0067] Exemplarily, Figure 4 It is a schematic diagram of facial key points provided by an embodiment of the present application. Figure 4 The number of the shown facial key points is 33. 28 of the 33 key points appear on both sides of the face, totaling 14 groups of symmetric points, and the remaining 5 head key points (1, 2, 33, 19, 20) are located on the vertical center line of the face. The 28 key points include: 6 key points of the eyebrows (7, 3, 5 and 6, 4, 8), 10 key points of the eyes (13, 9, 15, 17, 11 and 16, 10, 14, 18, 12), 6 key points of the nose (21, 22, 23, 24, 25, 26), and 6 key points of the mouth (27, 28, 29, 30, 31, 32).

[0068] It should be noted that, Figure 4 The shown facial key points are only taken as an example. In some embodiments, the facial key point detection model can detect more or fewer facial key points. For example, key points of the mandible, bottom of the ear, etc. can be added.

[0069] In some embodiments, the electronic device can determine the head size and the sizes of each part of the face according to the above key points. For example, determine the head height according to key points 1 and 33, determine the head width of the ears according to key points 19 and 22, determine the left eye width according to key points 13 and 15, determine the right eye width according to key points 14 and 16, determine the width of the nasal base according to key points 23 and 24, determine the width of the nose tip according to key points 21 and 22, determine the width of the mouth according to key points 29 and 30, etc.

[0070] In some embodiments, the facial feature information includes the size information of each part of the portrait face.

[0071] S304. Determine the deformation parameters of each part of the face based on the facial feature information of the portrait in the first image.

[0072] After obtaining the facial feature information of the portrait in the first image, the electronic device can adaptively adjust the parameters of the facial structure of the portrait in the first image within a preset value range based on the facial feature information of the standard face, so as to determine the deformation parameters of each part of the portrait face.

[0073] The facial feature information of the standard face includes the facial key point information of the standard face and the standard size information of each part of the face. The electronic device can pre-store the facial feature information of the standard face for facial structure optimization.

[0074] The preset numerical range includes the numerical ranges for adjusting the deformation parameters of various parts of the face. For example, the preset numerical range of the deformation parameter of the nose width, the preset numerical range of the deformation parameter of the eye width, and the preset numerical range of the deformation parameter of the mouth thickness. The preset numerical range defines the adjustment range of the deformation parameters of various parts of the face. It should be understood that if the deformation parameter is too large, it may cause facial distortion, and if the deformation parameter is too small, the optimization effect of the facial structure cannot be achieved.

[0075] There can be one or more standard faces. There are many types of standard faces, such as square faces, oval faces, diamond faces, elliptical faces, round faces, long faces, etc. The facial feature information of different standard faces is different. The electronic device can pre-store the facial feature information of multiple standard faces for facial structure optimization. The embodiments of the present application do not limit the data source of the standard face. Multiple standard faces can be generated based on an artificial intelligence AI model, or multiple standard faces can be obtained from a cloud database.

[0076] It should be understood that based on the facial feature information of different standard faces, the deformation parameters of various parts of the human face in the first image are determined, and the optimization effect of the facial structure is different.

[0077] In some embodiments, the facial feature information further includes facial contour information, and the facial contour information can be determined based on facial key points. The electronic device can confirm the face shape based on the facial contour information in the facial feature information of the human face in the first image, and then obtain the facial feature information of the standard face corresponding to the face shape. Then, based on the facial feature information of the standard face, within the preset numerical range, the facial structure of the human face in the first image is adaptively adjusted to determine the deformation parameters of various parts of the human face.

[0078] In some embodiments, the electronic device can determine the face shape of the human face in the first image based on a face similarity model, and then obtain the facial feature information of the standard face corresponding to the face shape. Then, based on the facial feature information of the standard face, within the preset numerical range, the facial structure of the human face in the first image is adaptively adjusted to determine the deformation parameters of various parts of the human face.

[0079] The face similarity model can determine the face shape similarity value and the facial feature similarity of the human face in the first image with multiple standard faces, and then determine the total similarity of the human face with multiple standard faces, and use the face shape of the standard face with the largest total similarity as the face shape of the human face in the first image. The facial feature similarity includes eye similarity, mouth similarity, nose similarity, eyebrow similarity, and ear similarity. The facial feature similarity can be the average value of the similarities of the above-mentioned various parts of the face. The total similarity can be the average value of the face shape similarity and the facial feature similarity.

[0080] The different parts of different human faces that differ from the standard face are different. The electronic device can determine the deformation parameters of all or part of the facial parts based on the differences between the human face and the standard face. Compared with the existing beauty templates that use a fixed set of deformation parameters, this method can achieve differential parameter adjustment of the facial structure of the human face, and the facial structure optimization effect is better.

[0081] Exemplarily, Figure 5 is a schematic diagram of setting the deformation parameters of the facial structure of the human face provided by the embodiment of the present application. Based on the key point information of the human face in the image, the size information of each part of the face can be determined. For example, Figure 5 in the figure, x1 represents the width of the left eye, x2 represents the distance between the center of the left eye and the center of the right eye, x3 represents the width of the nasal base, x4 represents the width of the mouth, y1 represents the height of the head, and y2 represents the distance from the chin to the horizontal center line of the mouth, denoted as the chin distance. Figure 5 This is only an example, and not all the facial dimensions are shown.

[0082] In one example, if the width of the nasal base of the human face is greater than the width of the nasal base of the standard face, the deformation parameter of the width of the nasal base can be set to a negative value to shorten the width of the nasal base of the human face. For example, Figure 5 in the figure, within the nasal base area, the width of the nasal base of the human face can be shortened based on the deformation parameter of the width of the nasal base. This example adjusts the width of the nasal base of the human face within the nasal base area to avoid affecting adjacent parts.

[0083] In one example, if the width of a single eye of the human face is less than the width of a single eye of the standard face, the deformation parameter of the width of the single eye can be set to a positive value to elongate the width of the single eye of the human face. For example, Figure 5 in the figure, within the left eye area, the width of the left eye of the human face can be elongated based on the deformation parameter of the width of the single eye. In the same way, the width of the right eye of the human face can be correspondingly elongated to make the left and right eyes symmetrical. This example adjusts the width of the left eye within the left eye area and adjusts the width of the right eye within the right eye area to avoid affecting adjacent parts. It should be understood that when adjusting the widths of the left and right eyes, the distance between the left and right eyes (such as x2) should also be considered to make the two eyes naturally distributed on the human face.

[0084] In one example, if the height of the chin of the human face is basically the same as the height of the chin of the standard face, the deformation parameter of the height of the chin can be set to 0, and 0 means that the height of the chin of the human face is not adjusted.

[0085] It should be understood that the adjustment principle of the deformation parameters of other parts of the human face is similar to the above several examples.

[0086] S305. Based on the deformation parameters of each part of the face, adjust the facial structure of the human face in the first image to obtain the second image.

[0087] By comparing the portrait with the standard face, the deformation parameters of the parts of the face to be adjusted can be determined. The electronic device can adjust the structures of the parts of the portrait face in the first image in regions based on the deformation parameters of the parts of the face, and obtain a second image. For example, the first image is Figure 2 the image shown in a in Figure 2 and the second image is the image shown in c in

[0088] The second image is an image after optimizing the facial structure of the portrait in the first image. When adjusting a certain part of the portrait face, the adjustment can be performed within the region where the part is located, which can avoid affecting adjacent parts.

[0089] The image processing method shown in the above embodiments identifies the facial feature information of the portrait in the image collected by the camera, determines the deformation parameters of the parts of the face based on the facial feature information of the portrait in the image and the facial feature information of the standard face, and optimizes the facial structure of the portrait in regions based on the deformation parameters of the parts. Compared with using a beauty template with fixed deformation parameters, this method can adaptively adjust the portrait face in a targeted manner, avoid facial structure distortion, and meet the personalized beauty needs of different users.

[0090] After completing the optimization of the facial structure of the portrait, makeup parameters can be configured on the optimized facial structure. The makeup parameters include contouring parameters, etc. Considering the differences in face shapes of different portraits, contouring parameters can be configured based on the category of the face shape, that is, different face shapes correspond to different contouring parameters. The electronic device can determine the face shape based on the facial feature information of the portrait and match the contouring parameters corresponding to the face shape to enhance the beauty effect of the portrait.

[0091] Exemplarily, Figure 6 is a schematic flow chart of the image processing method provided by the embodiment of the present application Figure 2 . Figure 6 The image processing method shown mainly involves the above second processing process. The method includes:

[0092] S601. Determine the face shape of the portrait based on the facial feature information of the portrait in the first image.

[0093] In one example, based on the key point information of the portrait face in the first image, the facial contour information of the portrait is determined; based on the facial contour information, the face shape of the portrait face is determined. In another example, through a portrait similarity model, the face shape similarity and the facial feature similarity between the portrait face in the first image and multiple standard faces are determined, the total similarity between the portrait face and multiple standard faces is determined, and the face shape of the standard face with the maximum total similarity is used as the face shape of the portrait face in the first image.

[0094] The above two examples are similar to the foregoing embodiments. For details, refer to the S304 part of the foregoing embodiments, which will not be elaborated here.

[0095] S602. Obtain the contouring parameters corresponding to the portrait face shape from the database.

[0096] The database of the electronic device pre-stores the contouring parameters corresponding to different face shapes, and the electronic device obtains the contouring parameters corresponding to the portrait face shape in the first image from the database. The contouring parameters include the positions, shapes, and degrees of facial highlights and shadows.

[0097] In some embodiments, the contouring parameters may be a contouring mask, and the contouring mask can indicate the positions, shapes, and degrees of facial highlights and shadows. The contouring mask can be regarded as a layer template, and this layer template can be superimposed on the portrait face area in the image to achieve portrait face contouring. The contouring mask can also be referred to as a contouring mask image.

[0098] In some embodiments, the database of the electronic device pre-stores the contouring mask images corresponding to different face shapes, and the electronic device obtains the contouring mask image corresponding to the portrait face shape in the first image from the database.

[0099] Exemplarily, Figure 7 is a schematic diagram of the contouring masks corresponding to different face shapes provided by the embodiments of the present application. Figure 7 From a to f in it, the contouring masks of a square face, an oval face, a diamond face, an oval face, a round face, and a rectangular face are shown in sequence. It can be seen that Figure 7 the facial highlight and shadow areas of different face shapes are different.

[0100] In some embodiments, based on the contouring methods corresponding to different face shapes, contouring masks corresponding to different face shapes can be generated to optimize the makeup of the portrait face and improve the portrait beauty effect.

[0101] In some embodiments, the contouring mask is used to indicate the positions, shapes, and degrees of the facial highlight area and the shadow area. The shapes include but are not limited to a water droplet shape, a lightning shape, a comb shape, a semi-arc shape, a "Z" shape, an "L" shape, etc.

[0102] Next, in combination with Figure 1The portraits in the examples will elaborate in detail on the contouring methods corresponding to different face shapes. Exemplarily, Figures 8a to 8f A schematic diagram showing the contouring methods corresponding to different face shapes.

[0103] As Figure 8a shown, if the portrait has an oval face shape, according to the characteristics of the oval face, highlight lines can be drawn at positions 3 to 6 on the face, and shadow lines can be drawn at positions 1 and 2 on the face. The highlight lines and shadow lines can be in the shape of a water droplet. For example, blend outward from both sides of the forehead corners in the shape of a water droplet to generate a shadow area.

[0104] As Figure 8b shown, if the portrait has a round face shape, according to the characteristics of the round face, highlight lines can be drawn at positions 5, 6, and 7 on the face, and shadow lines can be drawn at positions 1, 2, 3, and 4 on the face. The highlight lines and shadow lines can be in the shape of a lightning bolt. For example, blend outward from the cheekbones to the sides of the face to the jawline in the shape of a lightning bolt to generate a shadow area.

[0105] As Figure 8c shown, if the portrait has a rectangular face shape, according to the characteristics of the rectangular face, highlight lines can be drawn at positions 3 and 4 on the face, and shadow lines can be drawn at positions 1, 2, 5, 6, 7, and 8 on the face. The highlight lines and shadow lines can be in the shape of a comb "E". For example, blend outward from both sides of the cheekbones, both sides of the jaw, and both sides of the forehead corners in the shape of a comb to generate a shadow area.

[0106] As Figure 8d shown, if the portrait has a diamond face shape, according to the characteristics of the diamond face, highlight lines can be drawn at positions 3, 4, 7, and 8 on the face, and shadow lines can be drawn at positions 1, 2, 5, and 6 on the face. The highlight lines and shadow lines can be in the shape of a semi - arc "C". For example, blend inward from the edges at the cheekbones in the shape of a semi - arc to generate a shadow area.

[0107] As Figure 8e shown, if the portrait has an oval face shape, according to the characteristics of the oval face, highlight lines can be drawn at positions 7 to 10 on the face, and shadow lines can be drawn at positions 1 to 6 on the face. The highlight lines and shadow lines can be in the shape of a "Z". For example, blend outward from both sides of the cheekbones, both sides of the jaw, and both sides of the forehead corners in the shape of a "Z" to generate a shadow area.

[0108] As Figure 8f shown, if the portrait has a square face shape, according to the characteristics of the square face, highlight lines can be drawn at positions 7, 8, 11 to 14 on the face, and shadow lines can be drawn at positions 1 to 6, 9, 10, 15, and 16 on the face. The highlight lines and shadow lines can be in the shape of an "L". For example, blend outward from both sides of the cheekbones, both sides of the jaw, and both sides of the forehead corners in the shape of an "L" to generate a shadow area.

[0109] It should be noted that Figures 8a to 8fIn the image, if the contouring area is blocked by other objects, such as hair, the contouring area will not be superimposed with the corresponding highlight area or shadow area.

[0110] S603. Performing face retouching processing on the face of the portrait in the second image based on the face retouching parameters corresponding to the face shape of the portrait, to obtain a third image.

[0111] The electronic device obtains retouching parameters corresponding to the portrait face shape, and based on the facial highlight position and degree in the retouching parameters, superimposes a highlight area on the portrait face in the second image, and based on the facial shadow position and degree in the retouching parameters, superimposes a shadow area on the portrait face in the second image, to obtain a third image, in which highlights and shadows are superimposed on the portrait face.

[0112] For example, the second image is Figure 2 The image shown in c, the third image is Figure 2 The image shown in middle d.

[0113] In some embodiments, the contouring parameter corresponding to the portrait face shape may be a contouring mask. The electronic device may obtain a third image by performing image operation on the contouring mask and the second image. In one example, the electronic device determines that each pixel in the contouring mask corresponds to a pixel of the portrait face in the second image, and performs an AND operation on each pixel in the contouring mask and the pixel corresponding to the pixel in the second image to obtain the third image.

[0114] The image processing method shown in the above embodiment identifies the facial feature information of the portrait in the image captured by the camera, determines the face shape of the portrait, obtains the retouching parameters corresponding to the face shape of the portrait, and retouches the face of the portrait based on the retouching parameters corresponding to the face shape of the portrait to improve the portrait beautification effect. This method can perform retouching on the face of the portrait in a targeted manner, with different retouching areas and degrees for different face shapes, which can meet the personalized beautification needs of users with different face shapes.

[0115] In some embodiments, the electronic device may perform beauty processing on the facial parts of the portrait in the third image based on the features of the facial parts of the portrait to obtain a fourth image. The fourth image may be obtained by executing at least one of the following examples:

[0116] In one example, the electronic device obtains makeup parameters corresponding to the portrait eye features, such as eye bags parameters, catch light parameters, eye highlight parameters, eye shadow parameters, eyelash parameters, etc., based on the makeup parameters corresponding to the portrait eye features, and performs beauty makeup on the eyes of the portrait in the third image.

[0117] In one example, the electronic device obtains makeup parameters corresponding to the eyebrow features of the portrait, such as eyebrow dyeing parameters, based on the eyebrow features of the portrait, and performs beauty makeup processing on the eyebrows of the portrait in the third image based on the makeup parameters corresponding to the eyebrow features of the portrait.

[0118] In one example, the electronic device obtains makeup parameters corresponding to the portrait lip features, such as red lip parameters, based on the portrait lip features, and performs makeup processing on the portrait lips in the third image based on the makeup parameters corresponding to the portrait lip features.

[0119] In one example, the electronic device obtains makeup parameters corresponding to the portrait face features, such as blush parameters, based on the portrait face features, and performs makeup processing on the portrait face in the third image based on the makeup parameters corresponding to the portrait face features.

[0120] The image processing method shown in the above embodiments identifies the features of each part of the portrait face and performs targeted makeup processing on each part of the portrait face to achieve a better beauty effect.

[0121] The above electronic device may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The electronic device may be a mobile phone with a shooting and display function, a smart TV, a wearable device, a tablet computer (Pad), a computer with a wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. The embodiments of the present application do not limit the specific technologies and specific device forms adopted by the electronic device.

[0122] Exemplarily, Figure 9 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 9 shown, the electronic device 100 includes: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor 180, a button 190, a camera 193, and a display screen 194.

[0123] It can be understood that the structure illustrated in this embodiment does not specifically limit the electronic device 100. In some embodiments, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0124] It can be understood that the interface connection relationships between the modules illustrated in the embodiments are only illustrative and do not limit the structure of the electronic device 100. In some embodiments, the electronic device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0125] The processor 110 may include one or more processing units. Among them, different processing units may be independent devices or integrated in one or more processors. A memory may also be provided in the processor 110 for storing instructions and data.

[0126] The USB interface 130 is an interface that complies with the USB standard specification and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device, can also be used for data transmission between the electronic device and peripheral devices, and can also be used to connect headphones to play audio through the headphones.

[0127] The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110.

[0128] The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc. The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), NFC, infrared technology (IR), etc. applied to the electronic device 100.

[0129] The electronic device 100 can implement a display function through a GPU, a display screen 194, an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute instructions to generate or change display information.

[0130] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0131] The electronic device 100 can implement a shooting function through an image signal process (ISP) module, one or more cameras 193, a video codec, a GPU, one or more display screens 194, and an application processor, etc.

[0132] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. The camera 193 includes a lens, an image sensor (such as a CMOS image sensor (complementary metal oxide semiconductor image sensor, abbreviated as CIS)), a motor, etc.

[0133] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, data files such as music, photos, and videos are saved in the external memory card.

[0134] The internal memory 121 can be used to store one or more computer programs, and the one or more computer programs include instructions. The processor 110 can execute the above instructions stored in the internal memory 121, so that the electronic device 100 performs various functional applications and data processing, etc.

[0135] The sensor 180 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor 180K, an ambient light sensor, a bone conduction sensor, etc.

[0136] The touch sensor 180K, also known as a touch panel. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 together form a touch screen, also known as a touch control screen. The touch sensor 180K is used to detect touch operations acting thereon or nearby, and transmit the detected touch operations to the application processor to determine the type of touch event.

[0137] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys or touch keys. The electronic device 100 can receive key inputs and generate key signal inputs related to the user settings and function controls of the electronic device 100. For example, when the camera application is launched, the user can trigger the camera to take pictures or record videos by pressing the power-on key.

[0138] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the software system with a layered architecture is taken as an example of the Android system to exemplarily illustrate the software structure of the electronic device.

[0139] Figure 10 It is a schematic diagram of the software architecture of the electronic device provided by the embodiments of the present application. The layered architecture divides the software system of the electronic device into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. As Figure 10 shown, the electronic device includes an application layer, an application framework layer, a hardware abstraction layer, a driver layer, and a system service layer.

[0140] The application layer includes a camera application and third-party applications. The camera application is a system application, and the third-party applications include, but are not limited to, short video applications, camera applications, image processing applications, etc. The user can use the camera application to take pictures or videos, and the user can also use the third-party applications to call the camera to take pictures or videos. In some embodiments, the application package may further include applications such as a gallery, a calendar, a call, a map, a navigation, a Bluetooth, a music, a video, a short message, etc.

[0141] The application framework layer can provide application programming interfaces (APIs) and programming frameworks for the applications in the application layer. In the embodiments of the present application, the application framework layer includes a camera management module and a window management module. The camera management module is responsible for managing camera device information. The camera application can obtain camera characteristics, such as parameters like the number of cameras and shooting capabilities, through the camera management module. In some embodiments, the camera management module can also be used to transfer data between the camera application and the camera hardware abstraction layer. For example, the camera application transfers a notification message to turn on the beauty function to the camera hardware abstraction layer through the camera management module, so that after receiving the original image captured by the camera, the camera hardware abstraction layer can call the camera algorithm module to perform beauty processing on the portrait in the original image. The window management module is responsible for managing the windows in the application and the interaction with the user interface. For example, it is responsible for managing the camera application window and sending the window content (including the original image captured by the camera or the image after beauty processing) to the display driver for display.

[0142] The hardware abstraction layer is an interface layer located between the kernel layer and the hardware circuit. In the embodiments of the present application, the hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm module. The camera hardware abstraction layer can call the camera algorithm module to optimize the images or videos captured by the camera. In some embodiments, the camera algorithm module includes a first image processing module and a second image processing module. The first image processing module is used to detect the images captured by the camera, obtain the portrait facial feature information in the images, such as the portrait facial key point information, and based on the portrait facial feature information, determine the deformation parameters of each part of the face, and adjust the user's facial structure in the image based on the deformation parameters, and transfer the image with the adjusted facial structure to the second image processing module. The second image processing module is used to obtain the portrait facial feature information in the image from the first image processing module, determine the face shape based on the portrait facial feature information, match the corresponding contouring parameters, and perform contouring processing on the portrait face based on the contouring parameters to obtain the image after beauty processing. The parallel processing of the first image processing module and the second image processing module can improve the image processing speed.

[0143] It should be noted that the first image processing module and the second image processing module are not limited to the hardware abstraction layer. In some embodiments, the first image processing module and the second image processing module can also be located in the application layer. For example, the first image processing module and the second image processing module can be integrated into the camera application or a third-party application, or the camera application or a third-party application can call the first image processing module and the second image processing module in the application layer to implement the beauty function.

[0144] In some embodiments, the first image processing module and the second image processing module may also be integrated into one image processing module, which has the functions of the first image module and the second image processing module.

[0145] The driver layer provides drivers for different hardware devices. In the embodiments of the present application, the driver layer may include a camera driver and a display driver. The camera driver can be used to drive the camera of the electronic device to work to capture the original image. The display driver is used to drive the display screen of the electronic device to work to display the original image captured by the camera or the image after beauty processing.

[0146] Based on the above-mentioned software and hardware architecture of the electronic device, the internal execution process of the device of the image processing method provided by the embodiments of the present application will be described below in conjunction with a specific embodiment.

[0147] Exemplarily, Figure 11 is a schematic flow chart of the image processing method provided by the embodiments of the present application Figure 3 . Figure 11 Taking the case where the portrait shooting mode is turned on and the beauty function is not automatically turned on by the camera application as an example for the solution description. As Figure 11 shown, the image processing method may include the following steps:

[0148] S1101. In response to the operation of turning on the portrait shooting mode, the camera application sends a notification message of the portrait shooting mode to the camera management module.

[0149] Exemplarily, as Figure 1 shown, in response to the user's operation of selecting the portrait shooting mode in the shooting mode selection area 105 of the shooting interface 101, the camera application in the application layer sends a notification message of the portrait shooting mode to the camera management module in the application framework layer, and the camera management module can query the camera shooting parameters of the portrait shooting mode.

[0150] S1102. The camera management module sends the control information of the portrait shooting mode to the camera driver through the camera hardware abstraction layer.

[0151] The control information of the portrait shooting mode may include the camera shooting parameters of the portrait shooting mode, such as the identifier of the camera corresponding to the portrait shooting mode, and the focal length value of the camera, etc. The camera corresponding to the portrait shooting mode includes a telephoto camera.

[0152] S1103. The camera driver drives the corresponding camera to work and obtains the first image from the camera.

[0153] The camera driver drives the corresponding camera to work based on the camera shooting parameters in the control information of the portrait shooting mode. The first image is the original image captured by the camera, and the original image can be understood as an unprocessed image.

[0154] S1104. The camera driver sends the first image to the camera hardware abstraction layer.

[0155] When the beauty function of the camera application is not enabled, after receiving the first image sent by the camera driver, the camera hardware abstraction layer does not call the first image processing module and the second image processing module, that is, it does not perform image processing (such as beauty processing) on the first image. The camera hardware abstraction layer directly transmits the first image to the camera application through the camera management module, so that the camera application can send the first image to the display driver for display through the window management module. This process is the normal process of the portrait shooting mode, and the preview interface of the camera application displays the original image captured by the camera.

[0156] In some embodiments, the above image processing method further includes:

[0157] S1105. In response to an operation to enable the beauty function, the camera application sends a notification message to enable the beauty function to the camera management module. Exemplarily, as Figure 1 shown, in response to the user's operation of clicking the beauty function switch 103 in the preview area 102 of the shooting interface 101, the camera application sends a notification message to start the beauty function to the camera management module.

[0158] S1106. The camera management module sends a notification message to enable the beauty function to the camera hardware abstraction layer.

[0159] After receiving the first image (S1104) sent by the camera driver, the camera hardware abstraction layer executes S1107.

[0160] S1107. The camera hardware abstraction layer sends an image processing request to the first image processing module, and the image processing request includes the first image. The image processing request is used to trigger the first image processing module to adjust the facial structure of the first image.

[0161] S1108. The first image processing module identifies the facial feature information in the first image.

[0162] A facial recognition model and a facial key point detection model may be preset in the first image processing module. The first image processing module uses the facial recognition model to identify whether a human face is included in the first image. If it is recognized that a human face is included in the first image, the first image processing module further identifies the key points of each part of the human face through the facial key point detection model to obtain the facial feature information of the human face in the first image. The facial feature information includes the key point information of the human face.

[0163] In some embodiments, the facial feature information may further include facial contour information, and the facial contour information can be determined based on the facial key points.

[0164] S1109. The first image processing module determines the deformation parameters of each part of the face based on the facial feature information.

[0165] S1110. The first image processing module optimizes the facial structure based on the deformation parameters to obtain a second image. The second image is an image of the portrait in the first image with the facial structure optimized.

[0166] In this embodiment, S1109 and S1110 can respectively refer to S304 and S305 in the previous embodiment, and their implementation principles and effects are similar, so they will not be elaborated here.

[0167] S1111. The first image processing module sends the second image to the second image processing module.

[0168] After S1108, it further includes:

[0169] S1112. The first image processing module sends the facial feature information to the second image processing module.

[0170] S1113. The second image processing module determines the face shape based on the facial feature information.

[0171] S1114. The second image processing module obtains the contouring parameters corresponding to the face shape from the database.

[0172] S1115. The second image processing module performs contouring processing on the face of the second image based on the contouring parameters to obtain a third image. The third image is an image of the portrait in the second image with contouring processing.

[0173] In this embodiment, S1113 to S1115 can respectively refer to S601 to S603 in the previous embodiment, and their implementation principles and effects are similar, so they will not be elaborated here.

[0174] S1116. The second image processing module sends the third image to the camera application.

[0175] The second image processing module sends the third image to the camera hardware abstraction layer, the camera hardware abstraction layer sends the third image to the camera management module, and the camera management module sends the third image to the camera application.

[0176] S1117. The camera application sends the third image to the display driver to display the third image.

[0177] The camera application sends the third image to the window management module, and the window management module sends the third image to the display driver to display the third image. Exemplarily, as shown in b in Figure 1 , the beautified processed image, that is, the third image, is displayed in the preview area 108 of the shooting interface of the camera application.

[0178] The image processing method of this embodiment shows the interaction of each module inside the electronic device when the beauty function is turned on. The camera application realizes the optimization and contouring of the portrait facial structure of the original image by calling the underlying image processing modules of the electronic device, namely the first image processing module and the second image processing module, and presents the beautified portrait in the preview screen to meet the personalized beauty needs of different users.

[0179] In some embodiments, when the portrait shooting mode is turned on, the camera application can also automatically turn on the beauty function. The camera management module can query whether the automatic beauty function is configured in the portrait shooting mode after receiving the notification message of the portrait shooting mode of the camera application. If the automatic beauty function is configured in the portrait shooting mode, the control information of the portrait shooting mode can also include the notification message for turning on the beauty function, so that the camera hardware abstraction layer can call the first image processing module and the second image processing module for portrait detection and beauty processing (the aforementioned S1107 to S1115).

[0180] Based on the foregoing several embodiments, an embodiment of the present application proposes an image processing method applied to an electronic device. The method includes: in response to an operation of turning on the beauty function of the camera application, obtaining a first image including a portrait face collected by the camera; displaying a third image on the shooting interface of the camera application, where the third image is an image after the first image is superimposed with a contouring mask image, and the contouring mask image is a mask image matching the face shape of the portrait face, and the contouring mask image is used to indicate the positions, shapes, and degrees of the highlight area and the shadow area of the portrait face. The first image is the original image collected by the camera. The above method considers the differences in different portrait face shapes, superimposes a contouring mask image matching the face shape of the portrait face on the original image to meet the personalized beauty needs of different users and improve the portrait beauty effect.

[0181] In an alternative embodiment, before displaying the third image on the shooting interface of the camera application, the method further includes: obtaining the feature information of the portrait face in the first image; determining the face shape of the portrait face based on the feature information of the portrait face; obtaining a contouring mask image matching the face shape of the portrait face from the database, where the database includes contouring mask images corresponding to different face shapes; and performing a superimposing process on the first image and the contouring mask image to obtain the third image. The above method determines the portrait face shape by obtaining the portrait face features, obtains the contouring mask image corresponding to the portrait face shape, and improves the portrait facial contour in the image.

[0182] In an alternative embodiment, obtaining the feature information of the portrait face in the first image includes: determining the image area where the portrait face is located in the first image; and identifying the key points of each part of the portrait face in the first image through a pre-set facial key point detection model to obtain the key point information of the portrait face in the first image. The key point information includes the position information of the key points of each part of the portrait face. The above method detects through the facial key point detection model to obtain the key point information of each part of the portrait face, providing data support for face shape analysis, matching, and facial adjustment, etc.

[0183] In an alternative embodiment, determining the face shape of the portrait face based on the feature information of the portrait face includes: determining the facial contour information of the portrait based on the key point information of the portrait face in the first image; determining the face shape of the portrait face based on the facial contour information; or, determining the face shape similarity and facial feature similarity between the portrait face in the first image and multiple standard faces through a pre-set portrait similarity model, determining the total similarity between the portrait face and multiple standard faces, and taking the face shape of the standard face with the maximum total similarity as the face shape of the portrait face in the first image. The first method for determining the portrait face shape above determines the facial contour through portrait key points and matches the corresponding face shape based on the facial contour. The second method for determining the portrait face shape above is based on the portrait face similarity model, calculates the similarity between the portrait face shape and different standard face shapes, and then determines the portrait face shape.

[0184] In an alternative embodiment, the positions of the high-light areas and shadow areas of the portrait face in the contour mask maps corresponding to different face shapes are different, and the shapes of the high-light areas and shadow areas of the portrait face in the contour mask maps corresponding to different face shapes are different.

[0185] In an alternative embodiment, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask maps corresponding to different face shapes are different, including: if the face shape of the portrait face is an oval face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of a water droplet; or, if the face shape of the portrait face is a round face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of a lightning bolt; or, if the face shape of the portrait face is a rectangular face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of a comb; or, if the face shape of the portrait face is a diamond face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of a semi-arc; or, if the face shape of the portrait face is an elliptical face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of a Z; or, if the face shape of the portrait face is a square face, the shapes of the high-light areas and shadow areas of the portrait face in the contour mask map are in the shape of an L.

[0186] The above two embodiments illustrate the differences in the contour masking diagrams corresponding to different face shapes. After determining the face shape of a portrait, by matching the contour masking diagram corresponding to the face shape of the portrait, differential contouring of the portrait face can be achieved, enhancing the beauty effect of the portrait.

[0187] In an alternative embodiment, the method further includes: determining the deformation parameters of each part of the face based on the feature information of the portrait face; adjusting the structure of the portrait face in the first image region by region based on the deformation parameters of each part of the face to obtain a second image; performing an overlay process on the first image and the contour masking diagram, including: performing an overlay process on the second image and the contour masking diagram to obtain a third image. The above method first optimizes the structure of the portrait face, and on the basis of the optimized face structure, performs contouring on the portrait face to enhance the beauty effect of the portrait. It should be noted that when optimizing the structure of the portrait face, the structure of each part of the portrait face is adjusted region by region to avoid affecting adjacent parts.

[0188] In an alternative embodiment, determining the deformation parameters of each part of the face based on the feature information of the portrait face includes: adaptively adjusting the parameters of the structure of the portrait face in the first image within a preset value range based on the feature information of the portrait face and the facial feature information of a standard face, so as to determine the deformation parameters of each part of the portrait face. The deformation parameters of each part of the face should be set within a reasonable value range, that is, the deformation parameters of each part should be set within the preset value range corresponding to each part. It can be understood that if the deformation parameters are too large, it may cause facial distortion, and if the deformation parameters are too small, the optimization of the face structure is not obvious. By means of adaptive parameter adjustment, reasonable facial deformation parameters are generated to optimize the structure of the portrait face.

[0189] In an alternative embodiment, the standard face includes standard faces of multiple different face shapes; determining the deformation parameters of each part of the face based on the feature information of the portrait face and the facial feature information of the standard face, within a preset value range, by adaptively adjusting the parameters of the structure of the portrait face in the first image, so as to determine the deformation parameters of each part of the portrait face, includes: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face, within a preset value range, by adaptively adjusting the parameters of the structure of the portrait face in the first image, so as to determine the deformation parameters of each part of the portrait face. The above method combines the facial features of the standard face corresponding to the face shape of the portrait, adaptively adjusts the parameters of each part of the portrait face, generates reasonable facial deformation parameters, and optimizes the structure of the portrait face.

[0190] In an alternative embodiment, the method further includes: in response to a first operation of enabling the portrait shooting mode of the camera application, enabling the beauty effect function. Alternatively, in response to the first operation of enabling the portrait shooting mode, a first control is displayed on the shooting interface, and the first control is used to trigger the enabling or disabling of the beauty effect function; in response to a second operation on the first control, the beauty effect function is enabled.

[0191] Exemplarily, referring to Figure 1 , the first operation may be an operation in which the user selects the portrait shooting mode in the shooting mode selection area 105 of the shooting interface 101 of the camera application. In one example, in response to the first operation, the electronic device directly enables the beauty effect function. In another example, in response to the first operation, the camera application switches to the portrait shooting mode, and at this time, the beauty effect function is not enabled. Continuing to refer to Figure 1 , the first control may be the beauty effect function switch 103 in the preview area 102 of the shooting interface 101 of the camera application, and the second operation may be an operation of clicking the beauty effect function switch 103. In response to the second operation, the electronic device enables the beauty effect function. Different from the previous example, the beauty effect function needs to be manually enabled by the user.

[0192] The above method shows two ways to enable the beauty effect function. After enabling the beauty effect function, the electronic device may execute the above image processing method to meet the personalized beauty effect needs of different users.

[0193] It should be noted that the embodiments of the present application do not particularly limit the specific structure of the execution subject of an image processing method. As long as the code storing an image processing method of the embodiments of the present application can be run to perform processing according to the image processing method provided by the embodiments of the present application. For example, the execution subject of the image processing method provided by the embodiments of the present application may be a functional module in the electronic device that can call and execute a program, or a processing device applied to the electronic device. For example, the processing device is a chip.

[0194] In the above embodiments, the "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group of processors, etc.) for executing one or more software or firmware programs, a memory, a merged logic circuit, and / or other suitable components that support the described functions.

[0195] Therefore, the modules of the examples described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0196] The embodiments of the present application provide an electronic device, including: a memory and a processor, where the processor is configured to call a computer program in the memory to execute the technical solution of any of the foregoing method embodiments, and its implementation principle and technical effect are similar to those of the foregoing related embodiments, and will not be elaborated herein.

[0197] The memory can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0198] The memory can exist independently and be connected to the processor through a communication line. The memory can also be integrated with the processor.

[0199] The processor can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the solution of the present application.

[0200] The embodiments of the present application provide a computer-readable storage medium storing a computer program, and when the computer program runs on an electronic device, the electronic device is enabled to execute the technical solution of any of the foregoing embodiments, and its implementation principle and technical effect are similar to those of the foregoing related embodiments, and will not be elaborated herein.

[0201] An embodiment of the present application provides a chip, which includes a processor. The processor is used to call a computer program in a memory to execute the technical solution in any of the above embodiments. Its implementation principle and technical effects are similar to those of the above related embodiments, and will not be elaborated here.

[0202] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, it causes the electronic device to execute the technical solution in any of the above embodiments. Its implementation principle and technical effects are similar to those of the above related embodiments, and will not be elaborated here.

[0203] The above specific embodiments further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, applied to an electronic device, the method includes: responding to an operation of turning on the beauty function of the camera application, and obtaining a first image including a human face captured by the camera; displaying a third image on the shooting interface of the camera application, where the third image is an image obtained by superimposing a contour masking image on the first image, and the contour masking image is a masking image matching the face shape of the human face, and the contour masking image is used to indicate the positions, shapes and degrees of the high - light area and the shadow area of the human face.

2. The method according to claim 1, characterized in that, before displaying the third image on the shooting interface of the camera application, the method further includes: obtaining the feature information of the human face in the first image; determining the face shape of the human face based on the feature information of the human face; obtaining a contour masking image matching the face shape of the human face from a database, where the database includes contour masking images corresponding to different face shapes; performing a superimposing process on the first image and the contour masking image to obtain the third image.

3. The method according to claim 2, characterized in that, obtaining the feature information of the human face in the first image includes: determining the image area where the human face is located in the first image; identifying the key points of each part of the human face in the first image through a pre - set facial key - point detection model to obtain the key - point information of the human face in the first image.

4. The method according to claim 2 or 3, characterized in that, determining the face shape of the human face based on the feature information of the human face includes: determining the facial contour information of the human face based on the key - point information of the human face in the first image; determining the face shape of the human face based on the facial contour information; or determining the face - shape similarity and the facial - feature similarity between the human face in the first image and multiple standard faces through a pre - set human - face similarity model, determining the total similarity between the human face in the first image and the multiple standard faces, and taking the face shape of the standard face with the maximum total similarity as the face shape of the human face in the first image.

5. The method according to any one of claims 1 to 4, characterized in that, the positions of the high - light area and the shadow area of the human face in the contour masking images corresponding to different face shapes are different, and the shapes of the high - light area and the shadow area of the human face in the contour masking images corresponding to different face shapes are different.

6. The method according to claim 5, characterized in that, the shapes of the high - light area and the shadow area of the human face in the contour masking images corresponding to different face shapes are different, including: if the face shape of the human face is an oval face, the shapes of the high - light area and the shadow area of the human face in the contour masking image are in the shape of a water drop; or if the face shape of the human face is a round face, the shapes of the high - light area and the shadow area of the human face in the contour masking image are in the shape of a lightning bolt; or If the face shape of the portrait face is a rectangular face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask image are in the shape of a comb; or If the face shape of the portrait face is a diamond face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask image are in the shape of a semi - arc; or If the face shape of the portrait face is an oval face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask image are in the shape of a Z; or If the face shape of the portrait face is a square face, the shapes of the high - light area and the shadow area of the portrait face in the contour mask image are in the shape of an L.

7. The method according to any one of claims 1 to 6, wherein, the method further includes: determining deformation parameters of each part of the face based on the feature information of the portrait face; adjusting the structure of the portrait face in the first image in regions based on the deformation parameters of each part of the face to obtain a second image; performing an overlay process on the first image and the contour mask image to obtain the third image, including: performing an overlay process on the second image and the contour mask image to obtain the third image.

8. The method according to claim 7, wherein, determining deformation parameters of each part of the face based on the feature information of the portrait face includes: performing adaptive parameter adjustment on the structure of the portrait face in the first image within a preset value range based on the feature information of the portrait face and the facial feature information of a standard face to determine the deformation parameters of each part of the portrait face.

9. The method according to claim 8, wherein, the standard face includes standard faces of multiple different face shapes; performing adaptive parameter adjustment on the structure of the portrait face in the first image within a preset value range based on the feature information of the portrait face and the facial feature information of a standard face to determine the deformation parameters of each part of the portrait face includes: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; performing adaptive parameter adjustment on the structure of the portrait face in the first image within the preset value range based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face to determine the deformation parameters of each part of the portrait face.

10. The method according to any one of claims 1 to 9, wherein, the method further includes: responding to a first operation of turning on the portrait shooting mode of the camera application to turn on the beauty function; or responding to a first operation of turning on the portrait shooting mode, a first control is displayed on the shooting interface, and the first control is used to trigger the opening or closing of the beauty function; responding to a second operation acting on the first control to turn on the beauty function.

11. An electronic device, wherein, the electronic device includes: a memory and a processor, and the processor is used to call a computer program in the memory to execute the method according to any one of claims 1 to 10.

12. A chip, wherein, The chip includes a processor, and the processor is configured to call a computer program in a memory to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program runs on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 10.

Images