Image processing methods, devices and storage media

Abstract

This application provides an image processing method, apparatus, and storage medium, applied in the field of terminal technology. The method includes: when the beautification function of a camera application is enabled on an electronic device, the electronic device acquires an original image including a human face captured by the camera; after image processing, the beautified image is displayed on the camera application's shooting interface; the beautified image is an image of the original image superimposed with a contour mask, the contour mask being a mask matching the face shape of the human face, used to indicate the position, shape, and degree of highlight and shadow areas on the human face. The above method considers the differences in different human face shapes, superimposing a contour mask matching the face shape onto the original image to meet the personalized beautification needs of different users and improve the portrait beautification effect.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to image processing methods, devices and storage media. Background Technology

[0002] With the widespread use of smart electronic devices, users take photos using electronic devices such as mobile phones and tablets. Some users will use beauty functions to optimize face shape, adjust skin tone, and set makeup.

[0003] Currently, beauty filters are usually presented to users in the form of beauty templates. Users can choose their favorite beauty template from the beauty template library and take an image with a beauty effect.

[0004] Because the parameters in the beauty template, such as various facial deformation parameters and makeup parameters, are all fixed configuration parameters, it is impossible to achieve a personalized look for each user's different beauty needs. Summary of the Invention

[0005] This application provides an image processing method, device, and storage medium that can meet the personalized beautification needs of different users and improve the beautification effect of portraits.

[0006] In a first aspect, embodiments of this application propose an image processing method applied to an electronic device. The method includes: in response to activating a beauty function in a camera application, acquiring a first image captured by the camera, including a portrait face; displaying a third image on the camera application's shooting interface, the third image being an image after superimposing a contouring mask onto the first image. The contouring mask is a mask matching the face shape of the portrait face, used to indicate the position, shape, and degree of highlight and shadow areas on the portrait face. The first image is the original image captured by the camera. The above method considers the differences in different portrait face shapes, superimposing a contouring mask matching the face shape onto the original image to meet the personalized beauty needs of different users and improve the portrait beauty effect.

[0007] In an optional embodiment of the first aspect, before displaying the third image on the camera application's shooting interface, the method further includes: acquiring feature information of a person's face in the first image; determining the face shape of the person's face based on the feature information; acquiring a contouring mask image matching the face shape from a database, the database including contouring masks corresponding to different face shapes; and overlaying the first image and the contouring mask image to obtain the third image. The above method determines the face shape by acquiring facial features and obtains a contouring mask image corresponding to the face shape, thereby enhancing the facial contour in the image.

[0008] In one optional embodiment of the first aspect, obtaining feature information of a human face in a first image includes: determining the image region where the human face is located in the first image; and identifying key points of various parts of the human face in the first image using a preset facial key point detection model to obtain key point information of the human face in the first image. The key point information includes the positional information of key points of various parts of the human face. The above method obtains key point information of various parts of the human face through detection by a facial key point detection model, providing data support for face shape analysis, matching, and facial adjustment.

[0009] In one optional embodiment of the first aspect, determining the face shape of a person based on facial feature information includes: determining facial contour information of the person based on key point information of the person's face in a first image; determining the face shape of the person's face based on the facial contour information; or, determining the face shape similarity and facial feature similarity between the person's face in the first image and multiple standard faces using a preset face similarity model, determining the total similarity between the person's face and multiple standard faces, and taking the face shape of the standard face with the highest total similarity as the face shape of the person's face in the first image. The first method for determining the face shape described above determines the facial contour by using key points of the person's face and matches the corresponding face shape based on the facial contour. The second method for determining the face shape described above calculates the similarity between the face shape and different standard face shapes based on a face similarity model, thereby determining the face shape.

[0010] In one optional embodiment of the first aspect, the positions of the highlight and shadow areas of the face in the contour mask image corresponding to different face shapes are different, and the shapes of the highlight and shadow areas of the face in the contour mask image corresponding to different face shapes are different.

[0011] In one optional embodiment of the first aspect, the shapes of the highlight and shadow areas of the face in the contour mask image are different for different face shapes, including: if the face shape is oval, the shape of the highlight and shadow areas of the face in the contour mask image is teardrop-shaped; or, if the face shape is round, the shape of the highlight and shadow areas of the face in the contour mask image is lightning-shaped; or, if the face shape is rectangular, the shape of the highlight and shadow areas of the face in the contour mask image is comb-shaped; or, if the face shape is diamond-shaped, the shape of the highlight and shadow areas of the face in the contour mask image is semi-circular; or, if the face shape is elliptical, the shape of the highlight and shadow areas of the face in the contour mask image is Z-shaped; or, if the face shape is square, the shape of the highlight and shadow areas of the face in the contour mask image is L-shaped.

[0012] The two embodiments above illustrate the differences in contouring masks corresponding to different face shapes. After determining the face shape, by matching the contouring mask corresponding to the face shape, differentiated contouring of the face can be achieved, thereby improving the beautification effect of the face.

[0013] In an optional embodiment of the first aspect, the method further includes: determining deformation parameters of each part of the face based on the feature information of the face; adjusting the structure of the face in the first image by region based on the deformation parameters of each part of the face to obtain a second image; and superimposing the first image and the contouring mask to obtain a third image, including: superimposing the second image and the contouring mask to obtain the third image. The above method first optimizes the structure of the face, and then performs contouring on the face based on the optimized structure to improve the beautification effect. It is worth noting that when optimizing the structure of the face, the structure of each part of the face is adjusted by region to avoid affecting adjacent parts.

[0014] In one optional embodiment of the first aspect, the deformation parameters of each part of the face are determined based on the feature information of the human face. This includes: based on the feature information of the human face and the facial feature information of a standard face, adaptively adjusting the structure of the human face in the first image within a preset numerical range to determine the deformation parameters of each part of the human face. The deformation parameters of each part of the face should be set within a reasonable numerical range, that is, the deformation parameters of each part should be set within the preset numerical range corresponding to each part. It is understood that if the deformation parameters are too large, it may lead to facial distortion; if the deformation parameters are too small, the optimization of the facial structure will not be obvious. By adaptively adjusting the parameters, reasonable facial deformation parameters are generated to optimize the human face structure.

[0015] In one optional embodiment of the first aspect, the standard face includes various standard faces with different face shapes. Based on the feature information of the portrait face and the facial feature information of the standard face, within a preset numerical range, adaptive parameter tuning is performed on the structure of the portrait face in the first image to determine the deformation parameters of each part of the portrait face. This includes: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; and adaptively tuning the structure of the portrait face in the first image within a preset numerical range based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face to determine the deformation parameters of each part of the portrait face. The above method combines the facial features of the standard face corresponding to the portrait face shape to adaptively tune the parameters of each part of the portrait face to generate reasonable facial deformation parameters and optimize the structure of the portrait face.

[0016] In an optional embodiment of the first aspect, the method further includes: enabling a beautification function in response to a first operation of enabling a portrait shooting mode of a camera application. Alternatively, in response to the first operation of enabling a portrait shooting mode, a first control is displayed on the shooting interface, the first control being used to trigger enabling or disabling the beautification function; and the beautification function is enabled in response to a second operation acting on the first control.

[0017] For example, refer to Figure 1 The first operation could be the user selecting portrait shooting mode in the shooting mode selection area 105 of the camera application's shooting interface 101. In one example, in response to this first operation, the electronic device directly activates the beautification function. In another example, in response to the first operation, the camera application switches to portrait shooting mode, at which point the beautification function is not activated. (Continue to refer to...) Figure 1 The first control can be the beauty function switch 103 in the preview area 102 of the camera application shooting interface 101. The second operation can be clicking the beauty function switch 103. In response to the second operation, the electronic device turns on the beauty function. Unlike the previous example, the beauty function needs to be turned on by the user.

[0018] The above methods demonstrate two ways to enable the beauty function. After enabling the beauty function, the electronic device can perform the above image processing methods to meet the personalized beauty needs of different users.

[0019] Secondly, embodiments of this application provide an image processing apparatus, including: an acquisition module, configured to acquire a first image including a human face captured by a camera in response to an operation of activating the beautification function of a camera application; and a display module, configured to display a third image on the shooting interface of the camera application, wherein the third image is an image after superimposing a contouring mask on the first image, the contouring mask being a mask matching the face shape of the human face, and the contouring mask being used to indicate the position, shape, and degree of the highlight area and shadow area of ​​the human face.

[0020] Thirdly, embodiments of this application provide an electronic device, which includes a memory and a processor, the processor being configured to invoke a computer program in the memory to perform the method as described in any of the first aspects.

[0021] Fourthly, embodiments of this application provide a chip, the chip including a processor, the processor being configured to invoke a computer program in memory to perform the method as described in any of the first aspects.

[0022] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method described in any of the first aspects.

[0023] A sixth aspect is a computer program product comprising a computer program that, when run, causes a computer to perform the method described in any of the first aspects.

[0024] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding optional embodiments are similar, and will not be described again. Attached Figure Description

[0025] Figure 1 A schematic diagram illustrating interface changes of the electronic device provided in this application embodiment;

[0026] Figure 2 This is a general flowchart of the image processing method provided in the embodiments of this application;

[0027] Figure 3 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 ;

[0028] Figure 4 A schematic diagram of facial key points provided in the embodiments of this application;

[0029] Figure 5 A schematic diagram illustrating the setting of facial structure deformation parameters for a human face, provided in an embodiment of this application;

[0030] Figure 6 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 2 ;

[0031] Figure 7 This is a schematic diagram of contouring masks corresponding to different face shapes provided in the embodiments of this application;

[0032] Figure 8a A schematic diagram illustrating the contouring method for an oval face provided in this application embodiment;

[0033] Figure 8b A schematic diagram illustrating a contouring method for a round face provided in an embodiment of this application;

[0034] Figure 8c A schematic diagram illustrating a contouring method for a rectangular face provided in an embodiment of this application;

[0035] Figure 8d A schematic diagram illustrating a contouring method for a diamond-shaped face as provided in an embodiment of this application;

[0036] Figure 8e A schematic diagram illustrating a contouring method for an oval face provided in an embodiment of this application;

[0037] Figure 8fA schematic diagram illustrating a contouring method for a square face provided in an embodiment of this application;

[0038] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0039] Figure 10 A schematic diagram of the software architecture of the electronic device provided in the embodiments of this application;

[0040] Figure 11 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 3 . Detailed Implementation

[0041] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0042] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0043] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. "At least one of the following (kind / items)" or similar expressions refer to any combination of these items, including any combination of single (kind / items) or multiple (kind / items). For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0044] It should be noted that the user information (including but not limited to user device information, user personal information, user facial information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0045] When users take photos or videos using the application, the app provides beauty enhancement features. These features include various styles of filters, different levels of skin smoothing, whitening, makeup effects, face slimming, eye enlargement, nose slimming, and head reduction. After opening the application, for example, in a short video app, if a user wants to enhance their appearance while shooting a short video, they can tap on different beauty templates provided at the bottom of the screen to optimize their facial features and overall shooting style.

[0046] However, since the parameters in beauty templates are usually fixed configurations, it's impossible to achieve a truly personalized look for every user's unique needs. For example, if a user has large eyes, and the beauty template has a large eye deformation parameter, the user will experience severe eye distortion after using the template. Similarly, if a user has a round face, and the beauty template has a small chin contouring area, the user will not experience a slimming effect after using the template.

[0047] To address the aforementioned issues, this application provides an image processing method. When a beautification function is activated, the electronic device acquires facial feature information of the user in an image captured by a camera. Based on this facial feature information, it determines the deformation parameters of various parts of the user's face, achieving precise facial structure optimization. Furthermore, the electronic device can also determine the user's face shape based on the facial feature information. After optimizing the user's facial structure, it can further match corresponding contouring parameters based on the face shape to enhance the beautification effect. This method provides personalized beautification based on the facial features of different users, using different beautification parameters for each user, thus avoiding beautification distortion.

[0048] In this embodiment, the beautification parameters include deformation parameters of various parts of the user's face, makeup parameters, etc. The deformation parameters of various parts of the user's face include, but are not limited to, parameters for slimming the face, enlarging the eyes, slimming the nose, and reducing head size. The makeup parameters include contouring parameters, which include the position, shape, and intensity of facial highlights and shadows.

[0049] In some embodiments, the beautification parameters also include skin smoothing parameters, whitening parameters, and skin tone parameters.

[0050] In some embodiments, makeup parameters also include parameters for red lips, blush, eyebrow tint, under-eye bags, eye highlight, eye brightening, eyeshadow, and eyelashes.

[0051] Users can manually turn on the beauty mode switch in the camera app's shooting interface to activate the beauty mode. Alternatively, the camera app can automatically activate the beauty mode after the user selects portrait shooting mode in the shooting interface. This application does not limit the method of activating the beauty mode.

[0052] Portrait shooting mode can determine the background based on the portrait in the image and blur the background to create a background blur effect.

[0053] For example, Figure 1 This is a schematic diagram illustrating the interface changes of an electronic device provided in an embodiment of this application. Taking a mobile phone as an example, for instance... Figure 1 As shown in Figure a, when a user opens the camera application on their phone, they can select portrait shooting mode in shooting mode selection area 104. The preview area 102 of the shooting interface 101 can display the beauty function switch 103. At this time, the portrait in the preview area 102 is the original portrait captured by the camera. When the user clicks the switch 103 to enable the beauty function, and the phone detects a human face in the image captured by the camera, it can execute the image processing method provided in this application embodiment to achieve personalized beauty enhancement of the human face in the image, such as... Figure 1 As shown in b, the portrait in preview area 105 is a beautified portrait, such as the portrait being slimmed down, the lips are plumped, or the face is reshaped.

[0054] Understandable. Figure 1 The interface shown is merely an example of a possible interface style for an electronic device, such as a mobile phone, and should not be construed as limiting the embodiments of this application.

[0055] Combination Figure 1 Example, Figure 2 This paper illustrates the overall flow of the image processing method provided in the embodiments of this application. When the electronic device activates the beautification function, such as... Figure 2 As shown, the image processing process includes the following two processes:

[0056] The first processing step: The electronic device performs facial recognition on the image captured by the camera to obtain the user's facial feature information. Based on this facial feature information, it determines the deformation parameters of various parts of the face to adjust the user's facial structure. For example, Figure 2The user in question had a wider nasal base and a thinner upper lip. The initial treatment primarily involved minor adjustments to the nasal base and upper lip. It should be understood that different users have different facial features, and targeted adjustments can be made to specific areas of the user's face to avoid facial structural distortion and meet the aesthetic needs of different users.

[0057] The second processing step: The electronic device determines the user's face shape based on facial feature information, matches corresponding contouring parameters based on the user's face shape, and performs contouring processing on the user's face based on the contouring parameters to obtain a beautified image. For example, Figure 2 As shown in d, contouring the user's face mainly involves adding shadows and highlights. It should be understood that different users have different face shapes, so contouring can be tailored to enhance the overall effect.

[0058] The first and second processing procedures will be described in detail below.

[0059] For example, Figure 3 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 . Figure 3 The image processing method shown mainly involves the first processing step described above, which includes:

[0060] S301. Acquire the first image captured by the camera.

[0061] After the camera application is opened, the electronic device's camera can capture images in real time; these images are also called raw images. At any given moment, the image captured by the camera is the first image.

[0062] S302. Perform facial recognition on the first image to determine whether the first image contains a human face.

[0063] After receiving the first image captured by the camera, the electronic device can perform facial recognition on the first image through a preset facial recognition model to determine whether the first image contains a human face. If the first image contains a human face, the image area where the human face is located is identified, and S303 is executed.

[0064] S303. Obtain facial feature information of the human figure in the first image.

[0065] After determining the image region where the human face is located in the first image, the electronic device can identify the key points of each part of the human face in the first image through a preset facial key point detection model, so as to obtain the facial feature information of the human face in the first image, including the key point information of the human face.

[0066] The input to the facial landmark detection model can be the image region where the face of the person is located, as determined by the facial recognition model. Since the background region in the first image is removed from the model input, the processing speed of the facial landmark detection model can be improved.

[0067] For example, Figure 4 A schematic diagram of facial key points provided in an embodiment of this application. Figure 4 The number of facial keypoints shown is 33. 28 of these keypoints appear on both sides of the face, forming 14 sets of symmetrical points. The remaining 5 head keypoints (1, 2, 33, 19, 20) are located on the vertical center line of the face. The 28 keypoints include: 6 keypoints for the eyebrows (7, 3, 5 and 6, 4, 8), 10 keypoints for the eyes (13, 9, 15, 17, 11 and 16, 10, 14, 18, 12), 6 keypoints for the nose (21, 22, 23, 24, 25, 26), and 6 keypoints for the mouth (27, 28, 29, 30, 31, 32).

[0068] It should be noted that, Figure 4 The facial key points shown are merely examples. In some embodiments, the facial key point detection model can detect more or fewer facial key points, for example, key points such as the jawline and the base of the ear can be added.

[0069] In some embodiments, the electronic device can determine the head size and the dimensions of various facial parts based on the aforementioned key points. For example, the head height is determined based on key points 1 and 33, the head width of the ears is determined based on key points 19 and 22, the width of the left eye is determined based on key points 13 and 15, the width of the right eye is determined based on key points 14 and 16, the width of the nasal base is determined based on key points 23 and 24, the width of the nasal tip is determined based on key points 21 and 22, and the width of the mouth is determined based on key points 29 and 30, etc.

[0070] In some embodiments, facial feature information includes size information of various parts of a person's face.

[0071] S304. Based on the facial feature information of the human figure in the first image, determine the deformation parameters of each part of the face.

[0072] After acquiring the facial feature information of the human figure in the first image, the electronic device can adaptively adjust the facial structure of the human figure in the first image within a preset value range based on the facial feature information of a standard face, so as to determine the deformation parameters of each part of the human face.

[0073] The facial feature information of a standard face includes the key facial points and the standard dimensions of each part of the face. Electronic devices can pre-store the facial feature information of a standard face for facial structure optimization.

[0074] The preset value ranges include the adjustment ranges for deformation parameters of various facial parts, such as the preset value ranges for the deformation parameters of the nose tip width, the eye width, and the lip thickness. These preset value ranges limit the adjustment range of deformation parameters for each facial part. It should be understood that excessively large deformation parameters may lead to facial distortion, while excessively small deformation parameters may fail to achieve the desired facial structure optimization.

[0075] There can be one or more standard faces. There are various types of standard faces, such as square, oval, diamond, round, and oblong faces. Different standard faces have different facial feature information. Electronic devices can pre-store facial feature information of multiple standard faces for facial structure optimization. This application does not limit the data source for standard faces; multiple standard faces can be generated based on artificial intelligence (AI) models or obtained from cloud databases.

[0076] It should be understood that the optimization effect of facial structure varies depending on the facial feature information of different standard faces used to determine the deformation parameters of various parts of the face in the first image.

[0077] In some embodiments, facial feature information further includes facial contour information, which can be determined based on facial key points. The electronic device can identify the face shape based on the facial contour information in the facial feature information of the portrait in the first image, then obtain the facial feature information of a standard face corresponding to that face shape, and then, based on the facial feature information of the standard face, adaptively adjust the facial structure of the portrait in the first image within a preset numerical range to determine the deformation parameters of each part of the portrait's face.

[0078] In some embodiments, the electronic device can determine the face shape of the human figure in the first image based on a human figure similarity model, and then obtain the facial feature information of the standard face corresponding to the face shape. Based on the facial feature information of the standard face, the device can adaptively adjust the facial structure of the human figure in the first image within a preset numerical range to determine the deformation parameters of each part of the human figure's face.

[0079] A facial similarity model can determine the similarity values ​​of the face shape and facial features between a person in a first image and multiple standard faces, and then determine the overall similarity between the person's face and these standard faces. The face shape of the standard face with the highest overall similarity is taken as the face shape of the person in the first image. Facial feature similarity includes eye similarity, mouth similarity, nose similarity, eyebrow similarity, and ear similarity. The facial feature similarity can be the average of the similarity values ​​of each of these facial parts. The overall similarity can be the average of the face shape similarity and the facial feature similarity.

[0080] Different portraits differ from standard faces in different areas. Electronic devices can determine the deformation parameters of all or part of the face based on these differences. Compared to existing beauty templates that use a fixed set of deformation parameters, this method can achieve differentiated parameter adjustment of the facial structure, resulting in better facial structure optimization.

[0081] For example, Figure 5 This is a schematic diagram illustrating the setting of facial structural deformation parameters according to an embodiment of this application. Based on the key point information of the human face in the image, the size information of each part of the face can be determined, such as... Figure 5 In the diagram, x1 represents the width of the left eye, x2 represents the distance between the center of the left eye and the center of the right eye, x3 represents the width of the nasal base, x4 represents the width of the mouth, y1 represents the height of the head, and y2 represents the distance between the chin and the horizontal center line of the mouth, denoted as chin distance. Figure 5 For illustrative purposes only; full facial dimensions are not shown.

[0082] In one example, if the nasal base width of a portrait is greater than that of a standard face, the deformation parameter for the nasal base width can be set to a negative value to shorten the nasal base width of the portrait. Figure 5 In this example, the nasal base width of a portrait can be shortened within the nasal base region based on deformation parameters of the nasal base width. This example adjusts the nasal base width of a portrait within the nasal base region to avoid affecting adjacent areas.

[0083] In one example, if the width of a person's eye is smaller than the width of a standard face's eye, the deformation parameter for the eye width can be set to a positive value to lengthen the width of the person's eye. Figure 5 In this example, the width of the left eye can be increased based on a deformation parameter of the single eye width within the left eye area, and the width of the right eye can be increased in the same way to make the left and right eyes symmetrical. This example adjusts the width of the left eye within the left eye area and the width of the right eye within the right eye area to avoid affecting adjacent parts. It should be understood that when adjusting the width of the left and right eyes, the distance between the left and right eyes (e.g., x2) should also be considered to ensure that the eyes are naturally distributed on the face of the portrait.

[0084] In one example, if the chin height of the portrait is basically the same as the chin height of a standard face, the deformation parameter of the chin height can be set to 0, which means that the chin height of the portrait will not be adjusted.

[0085] It should be understood that the adjustment principle for the deformation parameters of other parts of the human face is similar to the above examples.

[0086] S305. Based on the deformation parameters of each part of the face, adjust the facial structure of the portrait in the first image to obtain the second image.

[0087] By comparing a human portrait with a standard face, the deformation parameters of the facial features to be adjusted can be determined. Based on these deformation parameters, the electronic device can adjust the structure of different parts of the face in the first image, region by region, to obtain a second image. For example, the first image might be... Figure 2 The image shown in figure a, the second image is Figure 2 Image c is shown in the first image. The second image is the image after facial structure optimization of the portrait in the first image. When adjusting a certain part of the portrait's face, adjustments can be made within the area where that part is located, thus avoiding the influence on adjacent parts.

[0088] For example, when adjusting the width of the nasal base of a portrait, the image area of ​​the nasal base width is first determined, and then the deformation parameter of the nasal base width is applied to the image area. Compared with applying the deformation parameter of the nasal base width to the entire facial area of ​​the portrait, this adjustment method will not affect other parts near the nasal base. For example, if there are nasolabial folds near the nasal base, applying the deformation parameter of the nasal base width to the entire facial area will cause the nasolabial folds to deform, resulting in facial distortion.

[0089] The image processing method described in the above embodiments identifies facial feature information of a person in an image captured by a camera. Based on the facial feature information of the person in the image and the facial feature information of a standard face, it determines the deformation parameters of each part of the face. Based on the deformation parameters of each part, it optimizes the facial structure of the person in different regions. Compared with using a beauty template containing fixed deformation parameters, this method can make targeted adaptive adjustments to the face of the person, avoiding facial structure distortion and meeting the personalized beauty needs of different users.

[0090] After optimizing the facial structure of a portrait, makeup parameters, including contouring parameters, can be configured on the optimized facial structure. Considering the differences in face shapes among different portraits, contouring parameters can be configured based on face shape categories, meaning different face shapes correspond to different contouring parameters. Electronic devices can determine the face shape based on the facial feature information of the portrait and match the corresponding contouring parameters to improve the portrait beautification effect.

[0091] For example, Figure 6 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 2 . Figure 6 The image processing method shown mainly involves the second processing step described above. This method includes:

[0092] S601. Determine the face shape of the portrait based on the facial feature information of the portrait in the first image.

[0093] In one example, facial contour information of the portrait is determined based on key point information of the face in the first image; the face shape of the portrait is then determined based on the facial contour information. In another example, a portrait similarity model is used to determine the face shape similarity and facial feature similarity between the portrait in the first image and multiple standard faces, and the total similarity between the portrait and multiple standard faces is determined. The face shape of the standard face with the highest total similarity is taken as the face shape of the portrait in the first image.

[0094] The two examples above are similar to the aforementioned embodiments, and can be found in section S304 of the aforementioned embodiments, which will not be repeated here.

[0095] S602. Retrieve the contouring parameters corresponding to the portrait face shape from the database.

[0096] The electronic device's database stores contouring parameters corresponding to different face shapes. The electronic device retrieves the contouring parameters corresponding to the face shape in the first image from the database. The contouring parameters include the position, shape, and intensity of facial highlights and shadows.

[0097] In some embodiments, the contouring parameters can be contouring masks, which indicate the position, shape, and degree of highlights and shadows on a person's face. A contouring mask can be viewed as a layer template; this layer template can be overlaid on the facial area of ​​an image to achieve facial contouring. A contouring mask can also be called a contouring mask image.

[0098] In some embodiments, the electronic device's database stores contour masks corresponding to different face shapes, and the electronic device retrieves the contour mask corresponding to the face shape in the first image from the database.

[0099] For example, Figure 7 This is a schematic diagram of contouring masks corresponding to different face shapes provided in the embodiments of this application. Figure 7 Contouring masks for square, oval, diamond, round, and rectangular faces are shown in sections a through f, respectively. Figure 7 It can be seen that the areas of highlight and shadow on the face are different for different face shapes.

[0100] In some embodiments, based on the contouring methods corresponding to different face shapes, contouring masks corresponding to different face shapes can be generated to optimize the makeup of the portrait face and enhance the portrait beautification effect.

[0101] In some embodiments, the contour mask is used to indicate the location, shape, and degree of highlight and shadow areas on the face. Shapes include, but are not limited to, teardrop shapes, lightning bolt shapes, comb shapes, semi-circular shapes, "Z" shapes, "L" shapes, etc.

[0102] The following is combined with Figure 1The example portrait illustrates in detail the contouring methods for different face shapes. For instance, Figures 8a to 8f The diagram illustrates contouring techniques for different face shapes.

[0103] like Figure 8a As shown, if the portrait has an oval face, highlights can be drawn at positions 3 to 6 on the face, and shadow lines can be drawn at positions 1 and 2, based on the characteristics of an oval face. The highlights and shadow lines can be teardrop-shaped. For example, a teardrop shape can be used to blend outwards from both sides of the forehead to create shadow areas.

[0104] like Figure 8b As shown, if the portrait has a round face, highlights can be drawn at positions 5, 6, and 7 on the face, and shadow lines can be drawn at positions 1, 2, 3, and 4, based on the characteristics of a round face. The highlights and shadow lines can be lightning bolt shapes. For example, a lightning bolt shape can be used to blend outwards from the cheekbone to the side of the face and then to the jawline to create shadow areas.

[0105] like Figure 8c As shown, if the portrait has a rectangular face, highlights can be drawn at positions 3 and 4 on the face, and shadow lines can be drawn at positions 1, 2, 5, 6, 7, and 8 on the face, based on the characteristics of a rectangular face. The highlights and shadow lines can be in the shape of a comb "E". For example, the shadows can be created by blending outwards in a comb shape from both sides of the cheekbones, both sides of the jaw, and both sides of the forehead.

[0106] like Figure 8d As shown, if the portrait has a diamond-shaped face, highlights can be drawn at positions 3, 4, 7, and 8 on the face, and shadow lines can be drawn at positions 1, 2, 5, and 6, based on the characteristics of a diamond-shaped face. The highlights and shadow lines can be semi-circular "C" shapes. For example, the cheekbone area can be shaded inward from the edge in a semi-circular shape to create a shadow area.

[0107] like Figure 8e As shown, if the portrait has an oval face, highlights can be drawn at positions 7 to 10 on the face, and shadow lines can be drawn at positions 1 to 6, based on the characteristics of an oval face. The highlights and shadow lines can be in a "Z" shape. For example, blend outwards in a "Z" shape from both sides of the cheekbones, both sides of the jaw, and both sides of the temples to create shadow areas.

[0108] like Figure 8f As shown, if the portrait has a square face, highlights can be drawn at positions 7, 8, 11 to 14 on the face, and shadow lines can be drawn at positions 1 to 6, 9, 10, 15, and 16 on the face, based on the characteristics of a square face. The highlights and shadow lines can be "L" shaped. For example, blend outwards in an "L" shape from both sides of the cheekbones, both sides of the jaw, and both sides of the temples to create shadow areas.

[0109] It should be noted that, Figures 8a to 8fIn the process, if the contouring area is obscured by other objects, such as hair, the corresponding highlight or shadow area will not be superimposed on that contouring area.

[0110] S603. Based on the contouring parameters corresponding to the human face shape, perform contouring processing on the human face in the second image to obtain the third image.

[0111] The electronic device acquires the contouring parameters corresponding to the human face shape, and based on the position and degree of the facial highlight in the contouring parameters, overlays a highlight area on the human face in the second image, and based on the position and degree of the facial shadow in the contouring parameters, overlays a shadow area on the human face in the second image to obtain a third image, in which the human face is overlaid with highlights and shadows.

[0112] For example, the second image is Figure 2 The image shown in C is the third image. Figure 2 The image shown in d.

[0113] In some embodiments, the contouring parameters corresponding to the facial features can be a contouring mask. The electronic device can obtain a third image by performing image operations on the contouring mask and the second image. In one example, the electronic device determines that each pixel in the contouring mask corresponds to a pixel on the facial features in the second image, and performs a bitwise AND operation on each pixel in the contouring mask and the corresponding pixel in the second image to obtain the third image.

[0114] The image processing method described in the above embodiments identifies facial features in images captured by a camera, determines the face shape, obtains contouring parameters corresponding to the face shape, and then performs contouring on the face based on these parameters to enhance the beautification effect. This method allows for targeted facial contouring, with different contouring areas and degrees for different face shapes, thus meeting the personalized beautification needs of users with different face shapes.

[0115] In some embodiments, the electronic device may perform makeup processing on different parts of a person's face in a third image based on features of those parts, thereby obtaining a fourth image. The fourth image can be obtained by performing at least one of the following examples:

[0116] In one example, the electronic device obtains makeup parameters corresponding to the eye features of the portrait, such as under-eye bags parameters, eye light parameters, bright eye parameters, eyeshadow parameters, eyelash parameters, etc., and performs makeup processing on the eyes of the portrait in the third image based on the makeup parameters corresponding to the eye features of the portrait.

[0117] In one example, the electronic device obtains makeup parameters, such as eyebrow dyeing parameters, corresponding to the eyebrow features of a person based on the eyebrow features of the person, and performs makeup processing on the eyebrows of the person in the third image based on the makeup parameters corresponding to the eyebrow features of the person.

[0118] In one example, the electronic device obtains makeup parameters corresponding to the lip features of a person, such as red lip parameters, and performs makeup processing on the lips of the person in the third image based on the makeup parameters corresponding to the lip features of the person.

[0119] In one example, the electronic device obtains makeup parameters, such as blush parameters, corresponding to the facial features of the person, and performs makeup processing on the face in the third image based on the makeup parameters corresponding to the facial features of the person.

[0120] The image processing method shown in the above embodiments identifies the features of various parts of a person's face and performs targeted beautification processing on each part of the face to achieve a better beautification effect.

[0121] The aforementioned electronic devices can also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. These electronic devices can include mobile phones with camera and display functions, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, and wireless terminals in smart homes. This application does not limit the specific technologies or forms of the electronic devices used.

[0122] For example, Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 100 includes: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor 180, a button 190, a camera 193, and a display screen 194.

[0123] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In some embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0124] It is understood that the interface connection relationships between the modules illustrated in the embodiments are merely illustrative and do not constitute a structural limitation on the electronic device 100. In some embodiments, the electronic device 100 may also employ different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0125] The processor 110 may include one or more processing units. These processing units may be independent devices or integrated within one or more processors. The processor 110 may also include memory for storing instructions and data.

[0126] USB port 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc. USB port 130 can be used to connect a charger to charge electronic devices, to transfer data between electronic devices and peripheral devices, or to connect headphones for audio playback.

[0127] The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, and the charging management module 140 is connected to the processor 110.

[0128] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Mobile communication module 150 can provide wireless communication solutions for electronic device 100, including 2G / 3G / 4G / 5G. Wireless communication module 160 can provide wireless communication solutions for electronic device 100, including wireless local area networks (WLAN), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), NFC, and infrared (IR) technology.

[0129] Electronic device 100 can realize display functions through GPU, display screen 194, and application processor. GPU is a microprocessor for image processing, connected to display screen 194 and application processor. GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute instructions to generate or modify display information.

[0130] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0131] Electronic device 100 can perform shooting functions through an image signal processing (ISP) module, one or more cameras 193, a video codec, a GPU, one or more displays 194, and an application processor.

[0132] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. Camera 193 includes a lens, an image sensor (such as a CMOS image sensor (complementary metal oxide semiconductor image sensor, abbreviated as CIS)), a motor, etc.

[0133] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, data files such as music, photos, and videos can be stored on the external memory card.

[0134] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the aforementioned instructions stored in the internal memory 121, thereby enabling the electronic device 100 to perform various functional applications and data processing, etc.

[0135] Sensor 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors 180K, ambient light sensors, bone conduction sensors, etc.

[0136] The touch sensor 180K, also known as a touch panel, can be located on the display screen 194. The touch sensor 180K and the display screen 194 together form a touchscreen, also called a touch display. The touch sensor 180K detects touch operations applied to or near it and transmits the detected touch operations to the application processor to determine the type of touch event.

[0137] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100. For example, when the camera application is turned on, the user can trigger the camera to take a picture or record a video by pressing the power button.

[0138] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the Android system as an example of a layered architecture software system to illustrate the software structure of an electronic device.

[0139] Figure 10 This is a schematic diagram of the software architecture of an electronic device provided in an embodiment of this application. The layered architecture divides the software system of the electronic device into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. For example... Figure 10 As shown, the electronic device includes an application layer, an application framework layer, a hardware abstraction layer, a driver layer, and a system service layer.

[0140] The application layer includes a camera app and third-party applications. The camera app is a system application, while third-party applications include, but are not limited to, short video apps, camera apps, and image processing apps. Users can use the camera app to capture images or videos, or they can use third-party applications to access the camera to capture images or videos. In some embodiments, the application package may also include applications such as gallery, calendar, call, map, navigation, Bluetooth, music, video, and SMS.

[0141] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. In this embodiment, the application framework layer includes a camera management module and a window management module. The camera management module manages camera device information, allowing the camera application to obtain camera characteristics such as the number of cameras and shooting capabilities. In some embodiments, the camera management module can also be used to transmit data between the camera application and the camera hardware abstraction layer. For example, the camera application can transmit a notification message to the camera hardware abstraction layer via the camera management module to enable beautification, so that the camera hardware abstraction layer, upon receiving the original image captured by the camera, can invoke the camera algorithm module to perform beautification processing on the portrait in the original image. The window management module manages windows within the application and interactions with the user interface, such as managing the camera application window and sending the window content (including the original image captured by the camera or the beautified image) to the display driver for display.

[0142] The hardware abstraction layer (HAL) is an interface layer located between the kernel layer and the hardware circuitry. In this embodiment, the HAL includes a camera hardware abstraction layer and a camera algorithm module. The camera hardware abstraction layer can call the camera algorithm module to optimize images or videos captured by the camera. In some embodiments, the camera algorithm module includes a first image processing module and a second image processing module. The first image processing module detects images captured by the camera, acquires facial feature information such as facial key points, determines deformation parameters of various facial parts based on the facial feature information, adjusts the user's facial structure in the image based on the deformation parameters, and transmits the image with adjusted facial structure to the second image processing module. The second image processing module acquires facial feature information from the first image processing module, determines the face shape based on the facial feature information, matches corresponding contouring parameters, and performs contouring processing on the face to obtain a beautified image. The first and second image processing modules process in parallel, which can improve image processing speed.

[0143] It should be noted that the first image processing module and the second image processing module are not limited to the hardware abstraction layer. In some embodiments, the first image processing module and the second image processing module can also be located in the application layer. For example, the first image processing module and the second image processing module can be integrated into a camera application or a third-party application, or the camera application or the third-party application can implement the beautification function by calling the first image processing module and the second image processing module in the application layer.

[0144] In some embodiments, the first image processing module and the second image processing module may also be integrated into one image processing module, having the functions of both the first image module and the second image processing module.

[0145] The driver layer provides drivers for different hardware devices. In this embodiment, the driver layer may include a camera driver and a display driver. The camera driver can be used to drive the camera of an electronic device to work and capture raw images. The display driver is used to drive the display screen of the electronic device to work and display the raw images captured by the camera or the images after beautification processing.

[0146] Based on the above-described electronic device hardware and software architecture, the internal execution process of the image processing method provided in this application embodiment will be described below with reference to a specific embodiment.

[0147] For example, Figure 11 A flowchart illustrating the image processing method provided in the embodiments of this application. Figure 3 . Figure 11 The solution is explained using the example of enabling portrait shooting mode without the camera app automatically activating the beauty filter. Figure 11 As shown, the image processing method may include the following steps:

[0148] S1101. In response to the operation of activating portrait shooting mode, the camera application sends a portrait shooting mode notification message to the camera management module.

[0149] For example, such as Figure 1 As shown, in response to the user's operation of selecting the portrait shooting mode in the shooting mode selection area 105 of the shooting interface 101, the camera application in the application layer sends a notification message of the portrait shooting mode to the camera management module in the application framework layer. The camera management module can query the camera shooting parameters of the portrait shooting mode.

[0150] S1102. The camera management module sends control information for portrait shooting mode to the camera driver through the camera hardware abstraction layer.

[0151] The control information for portrait shooting mode can include camera shooting parameters for portrait shooting mode, such as the identifier of the camera corresponding to portrait shooting mode and the focal length value of the camera. The camera corresponding to portrait shooting mode includes telephoto cameras.

[0152] S1103. The camera driver drives the corresponding camera to work and acquires the first image from the camera.

[0153] The camera driver uses the camera shooting parameters from the control information of the portrait shooting mode to drive the corresponding camera to work. The first image is the raw image captured by the camera, which can be understood as an unprocessed image.

[0154] S1104. The camera driver sends the first image to the camera hardware abstraction layer.

[0155] When the camera application does not have its beautification function enabled, after receiving the first image from the camera driver, the camera hardware abstraction layer does not call the first and second image processing modules; that is, it does not perform any image processing (e.g., beautification) on the first image. The camera hardware abstraction layer directly transmits the first image to the camera application through the camera management module, so that the camera application can send the first image to the display driver for display through the window management module. This process is the standard procedure for portrait shooting mode, and the camera application's preview interface displays the raw image captured by the camera.

[0156] In some embodiments, the above image processing method further includes:

[0157] S1105. In response to the operation of enabling the beauty mode, the camera application sends a notification message to the camera management module to enable the beauty mode. For example, such as... Figure 1 As shown, in response to the user clicking the beauty function switch 103 in the preview area 102 of the shooting interface 101, the camera application sends a notification message to the camera management module to start the beauty function.

[0158] S1106. The camera management module sends a notification message to the camera hardware abstraction layer to enable the beauty mode.

[0159] After receiving the first image sent by the camera driver (S1104), the camera hardware abstraction layer executes S1107.

[0160] S1107. The camera hardware abstraction layer sends an image processing request to the first image processing module. The image processing request includes the first image. The image processing request is used to trigger the first image processing module to perform facial structure adjustment on the first image.

[0161] S1108. The first image processing module identifies facial feature information in the first image.

[0162] The first image processing module may have a pre-installed face recognition model and a facial key point detection model. The first image processing module uses the face recognition model to identify whether the first image contains a human face. If a human face is detected in the first image, the first image processing module uses the facial key point detection model to further identify the key points of each part of the human face in order to obtain the facial feature information of the human face in the first image. The facial feature information includes the key point information of the human face.

[0163] In some embodiments, facial feature information may further include facial contour information, which may be determined based on facial key points.

[0164] S1109. The first image processing module determines the deformation parameters of each part of the face based on facial feature information.

[0165] S1110. The first image processing module optimizes the facial structure based on deformation parameters to obtain a second image. The second image is the image obtained by optimizing the facial structure of the portrait in the first image.

[0166] In this embodiment, S1109 and S1110 can refer to S304 and S305 of the previous embodiment, respectively. Their implementation principles and effects are similar, and will not be elaborated here.

[0167] S1111. The first image processing module sends the second image to the second image processing module.

[0168] Following S1108, it also includes:

[0169] S1112. The first image processing module sends facial feature information to the second image processing module.

[0170] S1113. The second image processing module determines the face shape based on facial feature information.

[0171] S1114. The second image processing module obtains the contouring parameters corresponding to the face shape from the database.

[0172] S1115. The second image processing module performs facial contouring on the face in the second image based on the contouring parameters to obtain the third image. The third image is the image of the face in the second image after contouring.

[0173] In this embodiment, S1113 to S1115 can refer to S601 to S603 of the previous embodiment, and their implementation principle and effect are similar, so they will not be elaborated here.

[0174] S1116. The second image processing module sends a third image to the camera application.

[0175] The second image processing module sends a third image to the camera hardware abstraction layer, the camera hardware abstraction layer sends a third image to the camera management module, and the camera management module sends a third image to the camera application.

[0176] S1117. The camera application sends a third image to the display driver to display the third image.

[0177] The camera application sends a third image to the window management module, which then sends the third image to the display driver for display. For example, such as... Figure 1 As shown in b, the beautified image, i.e., the third image, is displayed in the preview area 108 of the camera application's shooting interface.

[0178] The image processing method in this embodiment illustrates the interaction between various modules inside the electronic device when the beauty function is enabled. The camera application calls the underlying image processing modules of the electronic device, namely the first image processing module and the second image processing module, to optimize and reshape the facial structure of the original image portrait and present the beautified portrait in the preview screen, thus meeting the personalized beautification needs of different users.

[0179] In some embodiments, when the portrait shooting mode is activated, the camera application can also automatically activate the beautification function. The camera management module can, upon receiving a notification message from the camera application regarding the portrait shooting mode, query whether the automatic beautification function is configured in the portrait shooting mode. If the automatic beautification function is configured in the portrait shooting mode, the control information for the portrait shooting mode may also include a notification message for activating the beautification function, so that the camera hardware abstraction layer can call the first image processing module and the second image processing module to perform portrait detection and beautification processing (as described in S1107 to S1115 above).

[0180] Based on the foregoing embodiments, this application proposes an image processing method applied to an electronic device. The method includes: in response to activating a beauty function in a camera application, acquiring a first image including a human face captured by the camera; displaying a third image on the camera application's shooting interface, the third image being an image after superimposing a contouring mask on the first image. The contouring mask is a mask matching the face shape of the human face, used to indicate the position, shape, and degree of highlight and shadow areas on the human face. The first image is the original image captured by the camera. The above method considers the differences in different human face shapes, superimposing a contouring mask matching the face shape on the original image to meet the personalized beauty needs of different users and improve the portrait beauty effect.

[0181] In one optional embodiment, before displaying the third image on the camera application's shooting interface, the method further includes: acquiring feature information of the human face in the first image; determining the face shape of the human face based on the feature information; acquiring a contouring mask image matching the face shape from a database, the database including contouring masks corresponding to different face shapes; and overlaying the first image and the contouring mask image to obtain the third image. The above method determines the face shape by acquiring human face features and obtains a contouring mask image corresponding to the face shape, thereby enhancing the human face contour in the image.

[0182] In one optional embodiment, obtaining feature information of a human face in the first image includes: determining the image region where the human face is located in the first image; and identifying key points of various parts of the human face in the first image using a preset facial key point detection model to obtain key point information of the human face in the first image. The key point information includes the positional information of key points of various parts of the human face. The above method obtains key point information of various parts of the human face through detection by the facial key point detection model, providing data support for face shape analysis, matching, and facial adjustment.

[0183] In one optional embodiment, determining the face shape of a person based on facial feature information includes: determining facial contour information of the person based on key point information of the person's face in a first image; determining the face shape of the person's face based on the facial contour information; or, determining the face shape similarity and facial feature similarity between the person's face in the first image and multiple standard faces using a preset face similarity model, determining the total similarity between the person's face and multiple standard faces, and taking the face shape of the standard face with the highest total similarity as the face shape of the person's face in the first image. The first method for determining the face shape described above determines the facial contour by using key points of the person's face and matches the corresponding face shape based on the facial contour. The second method for determining the face shape described above calculates the similarity between the person's face shape and different standard face shapes based on a face similarity model, thereby determining the face shape.

[0184] In one optional embodiment, the positions of the highlight and shadow areas of the face in the contour mask image corresponding to different face shapes are different, and the shapes of the highlight and shadow areas of the face in the contour mask image corresponding to different face shapes are different.

[0185] In one optional embodiment, the shapes of the highlight and shadow areas of the face in the contour mask image are different for different face shapes, including: if the face shape is oval, the highlight and shadow areas of the face in the contour mask image are teardrop-shaped; or, if the face shape is round, the highlight and shadow areas of the face in the contour mask image are lightning-shaped; or, if the face shape is rectangular, the highlight and shadow areas of the face in the contour mask image are comb-shaped; or, if the face shape is diamond-shaped, the highlight and shadow areas of the face in the contour mask image are semi-circular; or, if the face shape is elliptical, the highlight and shadow areas of the face in the contour mask image are Z-shaped; or, if the face shape is square, the highlight and shadow areas of the face in the contour mask image are L-shaped.

[0186] The two embodiments above illustrate the differences in contouring masks corresponding to different face shapes. After determining the face shape, by matching the contouring mask corresponding to the face shape, differentiated contouring of the face can be achieved, thereby improving the beautification effect of the face.

[0187] In one optional embodiment, the method further includes: determining deformation parameters of each part of the face based on the feature information of the face; adjusting the structure of the face in the first image by region based on the deformation parameters of each part of the face to obtain a second image; and superimposing the first image and the contouring mask to obtain a third image, including: superimposing the second image and the contouring mask to obtain the third image. The above method first optimizes the structure of the face, and then performs contouring on the face based on the optimized structure to improve the beautification effect. It is worth noting that when optimizing the face structure, the structure of each part of the face is adjusted by region to avoid affecting adjacent parts.

[0188] In one optional embodiment, based on the feature information of the human face, the deformation parameters of each part of the face are determined, including: based on the feature information of the human face and the facial feature information of a standard face, adaptively adjusting the structure of the human face in the first image within a preset numerical range to determine the deformation parameters of each part of the human face. The deformation parameters of each part of the face should be set within a reasonable numerical range, that is, the deformation parameters of each part should be set within the preset numerical range corresponding to each part. It is understood that if the deformation parameters are too large, it may lead to facial distortion; if the deformation parameters are too small, the optimization of the facial structure will not be obvious. By adaptively adjusting the parameters, reasonable facial deformation parameters are generated to optimize the human face structure.

[0189] In one optional embodiment, the standard face includes various standard faces with different face shapes. Based on the feature information of the portrait face and the facial feature information of the standard face, within a preset numerical range, the structure of the portrait face in the first image is adaptively adjusted to determine the deformation parameters of each part of the portrait face. This includes: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; and, based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face, within a preset numerical range, adaptively adjusting the structure of the portrait face in the first image to determine the deformation parameters of each part of the portrait face. The above method combines the facial features of the standard face corresponding to the portrait face shape to adaptively adjust the parameters of each part of the portrait face to generate reasonable facial deformation parameters and optimize the portrait face structure.

[0190] In one optional embodiment, the method further includes: enabling a beautification function in response to a first operation of enabling the portrait shooting mode of the camera application. Alternatively, in response to the first operation of enabling the portrait shooting mode, the shooting interface displays a first control, the first control being used to trigger enabling or disabling the beautification function; and enabling the beautification function in response to a second operation acting on the first control.

[0191] For example, refer to Figure 1 The first operation could be the user selecting portrait shooting mode in the shooting mode selection area 105 of the camera application's shooting interface 101. In one example, in response to this first operation, the electronic device directly activates the beautification function. In another example, in response to the first operation, the camera application switches to portrait shooting mode, at which point the beautification function is not activated. (Continue to refer to...) Figure 1 The first control can be the beauty function switch 103 in the preview area 102 of the camera application shooting interface 101. The second operation can be clicking the beauty function switch 103. In response to the second operation, the electronic device turns on the beauty function. Unlike the previous example, the beauty function needs to be turned on by the user.

[0192] The above methods demonstrate two ways to enable the beauty function. After enabling the beauty function, the electronic device can perform the above image processing methods to meet the personalized beauty needs of different users.

[0193] It should be noted that the embodiments of this application do not specifically limit the specific structure of the execution subject of an image processing method. As long as the image processing method provided by the embodiments of this application can be processed by running code storing such image processing method. For example, the execution subject of an image processing method provided by the embodiments of this application can be a functional module in an electronic device that can call and execute a program, or a processing device applied in an electronic device, such as a chip.

[0194] In the above embodiments, a "module" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuits, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.

[0195] Therefore, the modules of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] This application provides an electronic device, including a memory and a processor. The processor is used to call a computer program in the memory to execute the technical solution of any of the foregoing method embodiments. Its implementation principle and technical effects are similar to those of the above-mentioned related embodiments, and will not be repeated here.

[0197] The memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0198] The memory can exist independently and be connected to the processor via communication lines. Alternatively, the memory can be integrated with the processor.

[0199] The processor may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program in this application.

[0200] This application provides a computer-readable storage medium storing a computer program. When the computer program runs on an electronic device, it causes the electronic device to execute the technical solution of any of the above embodiments. The implementation principle and technical effect are similar to the above related embodiments, and will not be repeated here.

[0201] This application provides a chip, which includes a processor. The processor is used to call a computer program in a memory to execute the technical solutions in any of the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0202] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the technical solution in any of the above embodiments. Its implementation principle and technical effects are similar to those in the above related embodiments, and will not be repeated here.

[0203] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: In response to a first operation of activating the portrait shooting mode of the camera application, the beauty function is activated; or in response to the first operation of activating the portrait shooting mode, a first control is displayed on the shooting interface, the first control being used to trigger the activation or deactivation of the beauty function; in response to a second operation acting on the first control, the beauty function is activated. In response to the operation of activating the beauty function of the camera application, the first image including the human face captured by the camera is obtained; Based on the feature information of the human face in the first image, the face shape of the human face is determined; Obtain the facial feature information of the standard face corresponding to the face shape; Based on the feature information of the human face and the facial feature information of the standard face, within a preset value range, the structure of the human face in the first image is adaptively adjusted to determine the deformation parameters of each part of the human face. Based on the deformation parameters of each part of the face, the structure of the human face in the first image is adjusted by region to obtain the second image; Obtain a contour mask image matching the face shape of the portrait from a preset database. The database includes contour mask images corresponding to different face shapes. The positions and shapes of the highlight and shadow areas of the portrait face are different in the contour mask images corresponding to different face shapes. The contour mask image is used to indicate the position, shape, and degree of the highlight and shadow areas of the portrait face. The second image and the contour mask are overlaid to add the highlight and shadow information from the contour mask onto the second image, resulting in a third image. Based on the features of each part of the human face in the third image, obtain the makeup parameters corresponding to each part; Based on the makeup parameters corresponding to each part, makeup processing is performed on each part of the face in the third image to obtain the fourth image; The fourth image is displayed on the camera application's shooting interface; Among them, the shapes of the highlight and shadow areas of the portrait face in the contouring mask image are different for different face shapes, including: If the face shape of the portrait is an oval face, the shape of the highlight area and the shadow area of ​​the portrait face in the contouring mask is a teardrop shape; If the face shape of the portrait is a round face, the shape of the highlight area and the shadow area of ​​the portrait face in the contouring mask image is a lightning bolt shape; If the face shape of the portrait is rectangular, the shape of the highlight area and shadow area of ​​the portrait in the contouring mask is a comb shape; If the face shape of the portrait is a diamond-shaped face, the shape of the highlight area and the shadow area of ​​the portrait in the contouring mask is a semi-circular shape. If the face shape of the portrait is an oval, the shape of the highlight area and the shadow area of ​​the portrait in the contouring mask is a Z shape. If the face shape of the portrait is square, the shape of the highlight area and shadow area of ​​the portrait face in the contouring mask is L-shaped.

2. The method according to claim 1, characterized in that, Obtaining the feature information of the human face in the first image includes: Determine the image region where the human face is located in the first image; By using a pre-set facial key point detection model, key points of each part of the face in the first image are identified to obtain key point information of the face in the first image.

3. The method according to claim 2, characterized in that, Based on the feature information of the human face, the facial shape of the human face is determined, including: Based on the key point information of the human face in the first image, the facial contour information of the human face is determined; based on the facial contour information, the face shape of the human face is determined; or By using a preset portrait similarity model, the similarity of the portrait face in the first image with the face shape and facial features of multiple standard faces is determined, the total similarity of the portrait face with the multiple standard faces is determined, and the face shape of the standard face with the largest total similarity is taken as the face shape of the portrait face in the first image.

4. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to invoke a computer program in the memory to perform the method as described in any one of claims 1 to 3.

5. A chip, characterized in that, The chip includes a processor for calling a computer program in memory to perform the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 3.

Images