Image processing method, device, and storage medium

Abstract

Provided in the present application are an image processing method, a device, and a storage medium, which are applied to the technical field of terminals. The method comprises: when an electronic device enables a beautification function of a camera application, the electronic device acquiring a raw image which is collected by a camera and comprises a portrait face; and after image processing, displaying, in a photographing interface of the camera application, an image that has been subjected to beautification processing, wherein the image that has been subjected to beautification processing is an image obtained after a contouring mask image is superimposed onto the raw image, the contouring mask image is a mask image matching the facial form of the portrait face, and the contouring mask image is used for indicating the positions, shapes and degrees of a highlight area and shadow area on the portrait face. In the method, the differences between facial forms of different portraits are taken into consideration, and a contouring mask image matching the facial form of a portrait face is superimposed onto a raw image, so as to meet personalized beautification requirements of different users, thereby improving the portrait beautification effect.

Description

Image processing method, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on November 22, 2023, with application number 202311578854.3 and application name “Image processing method, device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to image processing methods, devices, and storage media. Background Art

[0003] With the popularization of smart electronic devices, users use electronic devices such as mobile phones and tablets to take photos. Some users will use beauty functions to optimize face shape, adjust skin color, set makeup, etc.

[0004] At present, the beautification function is usually presented to users in the form of beauty templates. Users can select their favorite beauty templates from the beauty template library and take an image with a beauty effect.

[0005] Since the parameters in the beauty template, such as facial deformation parameters and makeup parameters, are a set of fixed configuration parameters, it is impossible to adapt to the different beauty needs of different users.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide an image processing method, device, and storage medium that can meet the personalized beauty needs of different users and improve portrait beauty effects.

[0008] In a first aspect, an embodiment of the present application proposes an image processing method for use in an electronic device, the method comprising: in response to an operation to activate a beauty function of a camera application, obtaining a first image including a portrait face captured by a camera; and displaying a third image on the shooting interface of the camera application, the third image being an image obtained by superimposing a retouching mask image on the first image, the retouching mask image being a mask image that matches the face shape of the portrait face, and the retouching mask image being used to indicate the position, shape, and degree of highlight and shadow areas of the portrait face. The first image is the original image captured by the camera, and the above method takes into account the differences in face shapes of different portraits, and superimposes a retouching mask image that matches the face shape of the portrait face on the original image to meet the personalized beauty needs of different users and enhance the beauty effect of the portrait.

[0009] In an optional embodiment of the first aspect, before the camera application's shooting interface displays the third image, the method further includes: obtaining facial feature information of the person in the first image; determining the person's facial shape based on the facial feature information; obtaining a retouching mask image that matches the person's facial shape from a database, the database including retouching mask images corresponding to different facial shapes; and superimposing the first image and the retouching mask image to obtain the third image. The above method enhances the facial contours of the person in the image by obtaining facial features of the person to determine the person's facial shape and obtaining a retouching mask image corresponding to the person's facial shape.

[0010] In an optional embodiment of the first aspect, obtaining feature information of a person's face in a first image includes: determining the image region in the first image where the person's face is located; and identifying key points of various parts of the person's face in the first image using a preset facial key point detection model to obtain key point information of the person's face in the first image. The key point information includes positional information of the key points of various parts of the person's face. The above method obtains key point information of various parts of the person's face through detection using the facial key point detection model, providing data support for face shape analysis, matching, and facial adjustment.

[0011] In an optional embodiment of the first aspect, the face shape of the portrait face is determined based on the feature information of the portrait face, including: determining the facial contour information of the portrait based on the key point information of the portrait face in the first image; determining the face shape of the portrait face based on the facial contour information; or, determining the face shape similarity and facial feature similarity of the portrait face in the first image with multiple standard faces through a preset portrait similarity model, determining the total similarity between the portrait face and the multiple standard faces, and taking the face shape of the standard face with the greatest total similarity as the face shape of the portrait face in the first image. The above-mentioned first method for determining the face shape of a portrait is to determine the facial contour through the portrait key points, and match the corresponding face shape based on the facial contour. The above-mentioned second method for determining the face shape of a portrait is to calculate the similarity between the face shape of the portrait and different standard face shapes based on the portrait face similarity model, and then determine the face shape of the portrait.

[0012] In an optional embodiment of the first aspect, the positions of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different, and the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different.

[0013] In an optional embodiment of the first aspect, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different, including: if the face shape of the portrait face is an oval face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are water drop shapes; or, if the face shape of the portrait face is a round face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are lightning shapes; or, if the face shape of the portrait face is a rectangular face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are comb shapes; or, if the face shape of the portrait face is a diamond face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are semi-arc shapes; or, if the face shape of the portrait face is an elliptical face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are Z shapes; or, if the face shape of the portrait face is a square face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are L shapes.

[0014] The above two embodiments show the differences in the retouching mask images corresponding to different face shapes. After determining the face shape of the portrait, by matching the retouching mask image corresponding to the face shape of the portrait, differentiated retouching of the portrait face can be achieved, thereby improving the portrait beauty effect.

[0015] In an optional embodiment of the first aspect, the method further includes: determining the deformation parameters of various parts of the face based on the feature information of the portrait face; adjusting the structure of the portrait face in the first image by region based on the deformation parameters of various parts of the face to obtain a second image; and superimposing the first image and the retouching mask image to obtain a third image, including: superimposing the second image and the retouching mask image to obtain a third image. The above method first optimizes the facial structure of the portrait, and then, based on the optimized facial structure, retouches the portrait face to enhance the portrait beauty effect. It is worth noting that when optimizing the facial structure of the portrait, the structure of various parts of the portrait face is adjusted by region to avoid affecting adjacent parts.

[0016] In an optional embodiment of the first aspect, the deformation parameters of each facial part are determined based on the characteristic information of the portrait face, including: based on the characteristic information of the portrait face and the facial characteristic information of the standard face, adaptively adjusting the structure of the portrait face in the first image within a preset numerical range to determine the deformation parameters of each facial part. The deformation parameters of each facial part should be set within a reasonable numerical range, that is, the deformation parameters of each part should be set within the preset numerical range corresponding to each part. It is understood that if the deformation parameters are too large, facial distortion may occur, and if the deformation parameters are too small, the facial structure optimization is not obvious. By adaptively adjusting the parameters, reasonable facial deformation parameters are generated to optimize the facial structure of the portrait.

[0017] In an optional embodiment of the first aspect, the standard face includes a plurality of standard faces of different face shapes; based on the feature information of the portrait face and the facial feature information of the standard face, the structure of the portrait face in the first image is adaptively adjusted within a preset numerical range to determine the deformation parameters of various parts of the portrait face, including: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face, adaptively adjusting the structure of the portrait face in the first image within a preset numerical range to determine the deformation parameters of various parts of the portrait face. The above method combines the facial features of the standard face corresponding to the face shape of the portrait face to adaptively adjust the parameters of various parts of the portrait face to generate reasonable facial deformation parameters and optimize the face structure of the portrait.

[0018] In an optional embodiment of the first aspect, the method further includes: in response to a first operation of enabling a portrait shooting mode in a camera application, enabling a beauty feature. Alternatively, in response to the first operation of enabling the portrait shooting mode, a first control is displayed on the shooting interface, the first control being used to toggle the beauty feature on or off; and in response to a second operation on the first control, enabling the beauty feature.

[0019] Exemplarily, with reference to FIG1 , the first operation may be an operation in which a user selects a portrait shooting mode in the shooting mode selection area 105 of the camera application shooting interface 101. In one example, in response to the first operation, the electronic device directly turns on the beauty function. In another example, in response to the first operation, the camera application switches to the portrait shooting mode, and the beauty function is not turned on at this time. Continuing to refer to FIG1 , the first control may be the beauty function switch 103 in the preview area 102 of the camera application shooting interface 101, and the second operation may be an operation of clicking the beauty function switch 103. In response to the second operation, the electronic device turns on the beauty function. Unlike the previous example, the beauty function needs to be turned on by the user.

[0020] The above methods illustrate two methods for turning on the beauty function. After turning on the beauty function, the electronic device can execute the above image processing method to meet the personalized beauty needs of different users.

[0021] In the second aspect, an embodiment of the present application provides an image processing device, including: an acquisition module, used to obtain a first image including a portrait face captured by a camera in response to an operation of turning on the beauty function of a camera application; a display module, used to display a third image on the shooting interface of the camera application, the third image is an image after the first image is superimposed with a retouching mask image, the retouching mask image is a mask image that matches the facial shape of the portrait face, and the retouching mask image is used to indicate the position, shape and degree of the highlight area and shadow area of ​​the portrait face.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a memory and a processor, wherein the processor is used to call a computer program in the memory to execute any method described in the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute any method as described in the first aspect.

[0024] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the method as described in any one of the first aspects.

[0025] In a sixth aspect, a computer program product comprises a computer program, which, when executed, enables a computer to execute the method as described in any one of the first aspects.

[0026] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding optional embodiments are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic diagram of interface changes of an electronic device provided in an embodiment of the present application;

[0028] FIG2 is a general flow chart of the image processing method provided in an embodiment of the present application;

[0029] FIG3 is a flowchart of an image processing method according to an embodiment of the present application;

[0030] FIG4 is a schematic diagram of facial key points provided in an embodiment of the present application;

[0031] FIG5 is a schematic diagram of setting facial structure deformation parameters of a portrait provided by an embodiment of the present application;

[0032] FIG6 is a second flow chart of the image processing method provided in an embodiment of the present application;

[0033] FIG7 is a schematic diagram of contouring masks corresponding to different face shapes provided by an embodiment of the present application;

[0034] FIG8a is a schematic diagram of a contouring method for an oval face according to an embodiment of the present application;

[0035] FIG8 b is a schematic diagram of a contouring method for a round face according to an embodiment of the present application;

[0036] FIG8c is a schematic diagram of a contouring method for a rectangular face according to an embodiment of the present application;

[0037] FIG8 d is a schematic diagram of a contouring method for a diamond-shaped face according to an embodiment of the present application;

[0038] FIG8e is a schematic diagram of a contouring method for an oval face according to an embodiment of the present application;

[0039] FIG8f is a schematic diagram of a contouring method for a square face according to an embodiment of the present application;

[0040] FIG9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0041] FIG10 is a schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0042] FIG11 is a third flow chart of the image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] To facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean different.

[0044] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0045] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items (kind / individual)" or similar expressions refers to any combination of these items, including any combination of single items (kind / individual) or plural items (kind / individual). For example, at least one of a, b or c (kind / individual) can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0046] It should be noted that the user information (including but not limited to user device information, user personal information, user facial information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0047] When users use apps to take photos or videos, they can use beauty features. These features include filters with varying degrees of skin smoothing, whitening, makeup, face slimming, enlarging eyes, slimming noses, and reducing heads. After opening an app, for example, in a short video app, if users want to enhance their appearance while shooting a short video, they can tap on the different beauty templates provided at the bottom of the screen to optimize their facial features and overall shooting style.

[0048] However, since the parameters in beauty templates are typically fixed, they can't adapt to the varying beauty needs of different users. For example, if a user has large eyes and the eye deformation parameters in a beauty template are large, the user's eyes will be severely distorted after using this beauty template. Another example is if a user has a round face and the chin contouring area in a beauty template is small, the user won't see a good face-slimming effect after using this beauty template.

[0049] In response to the above-mentioned problems, an embodiment of the present application illustrates an image processing method. When the beautification function is activated, the electronic device obtains the facial feature information of the user in the image captured by the camera, and determines the deformation parameters of each part of the user's face based on the facial feature information to achieve accurate facial structure optimization. In addition, the electronic device can also determine the user's face shape based on the facial feature information. After completing the structural optimization of the user's face, it can also match the retouching parameters corresponding to the face shape based on the face shape to enhance the beautification effect. The above method performs personalized beautification based on the facial features of different users. Different users use different beauty parameters, which will not cause beauty distortion.

[0050] In the embodiment of the present application, beautification parameters include deformation parameters of various parts of the user's face, makeup parameters, etc. Deformation parameters of various parts of the user's face include, but are not limited to, parameters for thinning the face, enlarging the eyes, thinning the nose, and reducing the head. Makeup parameters include contouring parameters, which include the position, shape, and degree of facial highlights and shadows.

[0051] In some embodiments, the beauty parameters also include skin resurfacing parameters, whitening parameters, and skin color parameters.

[0052] In some embodiments, the makeup parameters also include lip parameters, blush parameters, eyebrow dyeing parameters, eye bags parameters, eye light parameters, eye highlight parameters, eye shadow parameters, eyelash parameters, etc.

[0053] The user can manually turn on the beautification function in the camera application's shooting interface to activate the beautification function. Alternatively, after the user selects portrait shooting mode in the camera application's shooting interface, the camera application automatically turns on the beautification function. The embodiments of this application do not limit the method for turning on the beautification function.

[0054] The portrait shooting mode can determine the background based on the portrait in the image and blur the background to make the background appear blurred, that is, background blur.

[0055] For example, FIG1 is a schematic diagram of the interface changes of an electronic device provided by an embodiment of the present application. Taking a mobile phone as an example, as shown in FIG1a, a user opens the camera application on the mobile phone and selects the portrait shooting mode in the shooting mode selection area 104. The preview area 102 of the shooting interface 101 may display a switch 103 for the beauty function. At this time, the portrait in the preview area 102 is the original portrait captured by the camera. The user clicks switch 103 to turn on the beauty function. When the mobile phone detects a portrait face in the image captured by the camera, it can execute the image processing method provided by the embodiment of the present application to achieve personalized beauty of the portrait face in the image. As shown in FIG1b, the portrait in the preview area 105 is the portrait after beauty processing, such as a thin nose, thick lips, or contouring.

[0056] It will be understood that the interface shown in FIG1 merely illustrates a possible interface style of an electronic device taking a mobile phone as an example, and should not constitute a limitation on the embodiments of the present application.

[0057] In conjunction with the example of FIG1 , FIG2 shows the overall process of the image processing method provided by an embodiment of the present application. When the beauty function of the electronic device is activated, as shown in FIG2 , the image processing process includes the following two processing steps:

[0058] First processing step: The electronic device performs facial recognition on the image captured by the camera to obtain facial feature information of the user in the image. Based on this facial feature information, the electronic device determines deformation parameters for various facial parts to adjust the user's facial structure. For example, the user in Figure 2 has a wide nose base and a thin upper lip. The first processing step primarily fine-tunes the nose base and upper lip. It should be understood that different users have different facial features, and targeted adaptive adjustments can be made to localized areas of the user's face to avoid distortion of the facial structure and meet the beauty needs of different users.

[0059] The second processing step: The electronic device determines the user's face shape based on the facial feature information, matches the corresponding retouching parameters based on the user's face shape, and performs retouching on the user's face based on the retouching parameters to obtain a beautified image. For example, as shown in Figure 2(d), the retouching of the user's face primarily adds shadows and highlights to the user's face. It should be understood that different users have different face shapes, and targeted retouching can be performed to enhance the retouching effect.

[0060] The first processing procedure and the second processing procedure are described in detail below.

[0061] For example, FIG3 is a flowchart of an image processing method according to an embodiment of the present application. The image processing method shown in FIG3 mainly involves the first processing process described above, and the method includes:

[0062] S301. Obtain a first image captured by a camera.

[0063] After the camera application is turned on, the camera of the electronic device can collect images in real time. The images collected by the camera are also called original images. At a certain moment, the image collected by the camera is the first image.

[0064] S302. Perform facial recognition on the first image to determine whether the first image contains a human face.

[0065] After receiving the first image captured by the camera, the electronic device can perform facial recognition on the first image through a preset facial recognition model to determine whether the first image contains a portrait face. If the first image contains a portrait face, the electronic device confirms the image area where the portrait face is located and executes S303.

[0066] S303. Obtain facial feature information of the portrait in the first image.

[0067] After determining the image area where the face of the portrait in the first image is located, the electronic device can identify the key points of various parts of the face of the portrait in the first image through a preset facial key point detection model to obtain facial feature information of the portrait in the first image, where the facial feature information includes key point information of the face of the portrait.

[0068] The input of the facial key point detection model can be the image area where the portrait face is located determined by the facial recognition model. Since the model input eliminates the background area in the first image, the processing speed of the facial key point detection model can be improved.

[0069] For example, Figure 4 is a schematic diagram of facial key points provided in an embodiment of the present application. The number of facial key points shown in Figure 4 is 33, 28 of the 33 key points appear on both sides of the face, totaling 14 groups of symmetrical points, and the remaining 5 head key points (1, 2, 33, 19, 20) are located on the vertical center line of the face. The 28 key points include: 6 key points of eyebrows (7, 3, 5 and 6, 4, 8), 10 key points of eyes (13, 9, 15, 17, 11 and 16, 10, 14, 18, 12), 6 key points of nose (21, 22, 23, 24, 25, 26), and 6 key points of mouth (27, 28, 29, 30, 31, 32).

[0070] It should be noted that the facial key points shown in Figure 4 are only an example. In some embodiments, the facial key point detection model can detect more or fewer facial key points, for example, the jaw key point, the ear base key point, etc. can be added.

[0071] In some embodiments, the electronic device can determine the size of the head and various facial parts based on the aforementioned key points. For example, the electronic device can determine the head height based on key points 1 and 33, the head width of the ears based on key points 19 and 22, the left eye width based on key points 13 and 15, the right eye width based on key points 14 and 16, the nose base width based on key points 23 and 24, the nose tip width based on key points 21 and 22, and the mouth width based on key points 29 and 30.

[0072] In some embodiments, the facial feature information includes size information of various parts of the face of the portrait.

[0073] S304. Determine deformation parameters of various facial parts based on facial feature information of the portrait in the first image.

[0074] After obtaining the facial feature information of the portrait in the first image, the electronic device can adaptively adjust the facial structure of the portrait in the first image within a preset numerical range based on the facial feature information of the standard face to determine the deformation parameters of various parts of the portrait's face.

[0075] The facial feature information of a standard face includes facial key point information of the standard face and standard size information of various facial parts. The electronic device can pre-store the facial feature information of the standard face for facial structure optimization.

[0076] The preset numerical ranges include the numerical ranges for adjusting the deformation parameters of various facial parts, such as the preset numerical ranges for the width of the nose tip, the width of the eyes, and the thickness of the mouth. These numerical ranges define the adjustment ranges for the deformation parameters of various facial parts. It should be understood that excessively large deformation parameters may cause facial distortion, while excessively small deformation parameters may not optimize the facial structure.

[0077] There can be one or more standard faces. There are many types of standard faces, such as square, oval, diamond, oval, round, and long. Different standard faces have different facial feature information. Electronic devices can pre-store facial feature information of multiple standard faces for facial structure optimization. The embodiment of this application does not limit the source of data for standard faces. Multiple standard faces can be generated based on an artificial intelligence (AI) model or obtained from a cloud database.

[0078] It should be understood that based on the facial feature information of different standard faces, the deformation parameters of various parts of the face of the portrait in the first image are determined, and the optimization effects of the facial structure are different.

[0079] In some embodiments, the facial feature information also includes facial contour information, which can be determined based on facial key points. The electronic device can determine the face shape based on the facial contour information in the facial feature information of the person in the first image, and then obtain facial feature information of a standard face corresponding to the face shape. Based on the facial feature information of the standard face, the electronic device can adaptively adjust the facial structure of the person in the first image within a preset value range to determine deformation parameters of various parts of the person's face.

[0080] In some embodiments, the electronic device may determine the face shape of the portrait in the first image based on a portrait similarity model, and then obtain facial feature information of a standard face corresponding to the face shape. Then, based on the facial feature information of the standard face, the electronic device may adaptively adjust the facial structure of the portrait in the first image within a preset numerical range to determine the deformation parameters of various parts of the portrait's face.

[0081] The portrait similarity model determines the face shape similarity and facial feature similarity between the portrait face in the first image and multiple standard faces, and then determines the overall similarity between the portrait face and the multiple standard faces. The face shape of the standard face with the greatest overall similarity is used as the face shape of the portrait face in the first image. Facial feature similarity includes eye similarity, mouth similarity, nose similarity, eyebrow similarity, and ear similarity. Facial feature similarity can be the average of the similarities of these facial parts. The overall similarity can be the average of the face shape similarity and facial feature similarity.

[0082] Different portraits differ from standard faces in different parts. The electronic device can determine the deformation parameters of all or part of the face based on the differences between the portrait and the standard face. Compared with the existing beauty templates that use a fixed set of deformation parameters, this method can achieve differentiated parameter adjustment of the portrait's facial structure, resulting in better facial structure optimization effects.

[0083] For example, Figure 5 is a schematic diagram of setting facial structural deformation parameters for a portrait, as provided in an embodiment of the present application. Based on the key facial point information in the image, the dimensions of various facial parts can be determined. In Figure 5, x1 represents the width of the left eye, x2 represents the distance between the center of the left eye and the center of the right eye, x3 represents the width of the nose base, x4 represents the width of the mouth, y1 represents the height of the head, and y2 represents the distance from the chin to the horizontal centerline of the mouth, denoted as the chin distance. Figure 5 is for illustrative purposes only and does not depict the full dimensions of the face.

[0084] In one example, if the nose base width of a portrait is wider than that of a standard face, the nose base width deformation parameter can be set to a negative value to shorten the portrait's nose base width. As shown in Figure 5, the nose base width of the portrait can be shortened based on the nose base width deformation parameter within the nose base area. This example adjusts the nose base width within the nose base area to avoid affecting adjacent areas.

[0085] In one example, if the width of one eye of a portrait is smaller than that of a standard face, the deformation parameter of the single eye width can be set to a positive value to lengthen the width of one eye of the portrait. As shown in Figure 5, the width of the left eye of the portrait can be lengthened based on the deformation parameter of the single eye width in the left eye area. The width of the right eye of the portrait can be lengthened in the same way to make the left and right eyes symmetrical. This example adjusts the width of the left eye in the left eye area and the width of the right eye in the right eye area to avoid affecting the adjacent parts. It should be understood that when adjusting the width of the left and right eyes, the distance between the left and right eyes (such as x2) should also be considered so that the eyes are naturally distributed on the face of the portrait.

[0086] In one example, if the chin height of the portrait is substantially the same as the chin height of a standard face, the deformation parameter of the chin height may be set to 0, where 0 indicates that the chin height of the portrait is not adjusted.

[0087] It should be understood that the adjustment principles of the deformation parameters of other parts of the portrait face are similar to the above examples.

[0088] S305. Based on the deformation parameters of each facial part, adjust the facial structure of the portrait in the first image to obtain a second image.

[0089] By comparing the portrait with a standard face, the deformation parameters of the facial area to be adjusted can be determined. Based on the deformation parameters of each facial area, the electronic device can adjust the structure of each facial area in the portrait in the first image, region by region, to produce a second image. For example, the first image is shown in Figure 2a, and the second image is shown in Figure 2c. The second image is the result of optimizing the facial structure of the portrait in the first image. When adjusting a facial area, the adjustment can be performed within the area where the area is located, thus avoiding affecting adjacent areas.

[0090] For example, when adjusting the nose base width of a portrait, the image area of ​​the nose base width is first determined, and then the deformation parameters of the nose base width are applied within the image area. Compared with applying the deformation parameters of the nose base width to the entire facial area of ​​the portrait, this adjustment method will not affect other parts near the nose base. For example, if there are nasolabial folds near the nose base, if the deformation parameters of the nose base width are applied to the entire facial area, the nasolabial folds will be deformed, causing facial distortion.

[0091] The image processing method illustrated in the above embodiment identifies facial features of a person in an image captured by a camera. Based on these features and those of a standard face, it determines deformation parameters for each facial component. Based on these deformation parameters, the facial structure of the person is optimized on a regional basis. Compared to using a beauty template with fixed deformation parameters, this method allows for targeted adaptive adjustments to the person's face, avoiding structural distortion and meeting the personalized beauty needs of different users.

[0092] After optimizing the facial structure of a portrait, you can configure makeup parameters based on the optimized facial structure. These parameters include contouring parameters. To account for the differences in facial shapes, contouring parameters can be configured based on face type, with different contouring parameters corresponding to different face shapes. The electronic device can determine the face shape based on the facial features of the portrait and match the contouring parameters to the face shape, thereby enhancing the portrait's beauty effect.

[0093] For example, FIG6 is a second flow chart of the image processing method provided in an embodiment of the present application. The image processing method shown in FIG6 mainly involves the second processing process described above, and the method includes:

[0094] S601. Determine the face shape of the portrait based on facial feature information of the portrait in the first image.

[0095] In one example, facial contour information of the portrait is determined based on key point information of the face in the first image; and the face shape of the portrait is determined based on the facial contour information. In another example, a portrait similarity model is used to determine the facial shape and facial feature similarity of the portrait in the first image with multiple standard faces, and the overall similarity between the portrait and the multiple standard faces is determined. The face shape of the standard face with the greatest overall similarity is used as the face shape of the portrait in the first image.

[0096] The above two examples are similar to the aforementioned embodiment. Please refer to S304 of the aforementioned embodiment for details, which will not be repeated here.

[0097] S602. Obtain contouring parameters corresponding to the portrait face shape from the database.

[0098] The electronic device has a database pre-stored with facial contouring parameters corresponding to different face shapes, and the electronic device obtains the facial contouring parameters corresponding to the face shape of the person in the first image from the database. The facial contouring parameters include the position, shape, and degree of facial highlights and shadows.

[0099] In some embodiments, the retouching parameters can be a retouching mask that indicates the location, shape, and extent of highlights and shadows on the face of the portrait. The retouching mask can be considered a layer template that can be overlaid on the facial area of ​​the portrait in the image to achieve facial retouching. The retouching mask can also be referred to as a retouching mask map.

[0100] In some embodiments, a database of the electronic device pre-stores retouching mask images corresponding to different face shapes, and the electronic device obtains the retouching mask image corresponding to the face shape of the portrait in the first image from the database.

[0101] For example, Figure 7 is a schematic diagram of the contouring masks corresponding to different face shapes provided in an embodiment of the present application. In Figure 7, a to f respectively show the contouring masks for a square face, an oval face, a diamond face, an oval face, a round face and a rectangular face. It can be seen from Figure 7 that the facial highlights and shadow areas of different face shapes are different.

[0102] In some embodiments, based on the contouring methods corresponding to different face shapes, contouring masks corresponding to different face shapes can be generated to optimize the makeup of the portrait face and enhance the portrait beauty effect.

[0103] In some embodiments, the contouring mask is used to indicate the location, shape, and extent of highlight and shadow areas on the face. Shapes include, but are not limited to, a teardrop shape, a lightning shape, a comb shape, a semi-arc shape, a "Z" shape, an "L" shape, and the like.

[0104] The following describes in detail the retouching methods corresponding to different face shapes in conjunction with the portrait in the example of Figure 1. For example, Figures 8a to 8f show schematic diagrams of the retouching methods corresponding to different face shapes.

[0105] As shown in Figure 8a, if the face shape is oval, based on the oval face, highlight lines can be drawn at positions 3 to 6 on the face, and shadow lines can be drawn at positions 1 and 2 on the face. The highlight and shadow lines can be in the shape of teardrops. For example, the shadow area can be created by blending outwards from the temples in a teardrop shape.

[0106] As shown in Figure 8b, if the face is round, highlight lines can be drawn at positions 5, 6, and 7, and shadow lines can be drawn at positions 1, 2, 3, and 4, based on the round face. The highlight and shadow lines can be in the shape of a lightning bolt. For example, a lightning bolt shape can be applied from the cheekbones to the sides of the face and then to the jawline, creating a shadow area.

[0107] As shown in Figure 8c, if the face is rectangular, highlight lines can be drawn at positions 3 and 4, and shadow lines can be drawn at positions 1, 2, 5, 6, 7, and 8, based on the rectangular face. The highlight and shadow lines can form a comb-shaped "E." For example, blend outward from the cheekbones, jaw, and temples in a comb-shaped pattern to create shadow areas.

[0108] As shown in Figure 8d, if the face shape is diamond-shaped, highlight lines can be drawn at positions 3, 4, 7, and 8, and shadow lines can be drawn at positions 1, 2, 5, and 6. The highlight and shadow lines can be in the shape of a semi-arc "C." For example, at the cheekbones, blend inward in a semi-arc shape from the edge to create a shadow area.

[0109] As shown in Figure 8e, if the face shape is oval, highlight lines can be drawn at positions 7 to 10 and shadow lines can be drawn at positions 1 to 6 based on the oval face. The highlight and shadow lines can be in a "Z" shape. For example, the shadow areas can be created by blending outwards from the cheekbones, the jaw, and the temples in a "Z" shape.

[0110] As shown in Figure 8f, if the face is square, highlight lines can be drawn at positions 7, 8, 11 to 14, and shadow lines can be drawn at positions 1 to 6, 9, 10, 15, and 16 based on the square face. The highlight and shadow lines can be in an "L" shape. For example, you can blend outward from the cheekbones, the jaw, and the temples in an "L" shape to create shadow areas.

[0111] It should be noted that in Figures 8a to 8f, if the trimming area is blocked by other objects, such as hair, the trimming area will not be superimposed with the corresponding highlight area or shadow area.

[0112] S603. Perform face retouching on the portrait in the second image based on the face retouching parameters corresponding to the face shape of the portrait to obtain a third image.

[0113] The electronic device obtains retouching parameters corresponding to the portrait face shape, and based on the facial highlight position and degree in the retouching parameters, superimposes a highlight area on the portrait face in the second image, and based on the facial shadow position and degree in the retouching parameters, superimposes a shadow area on the portrait face in the second image, to obtain a third image, in which the portrait face in the third image is superimposed with highlights and shadows.

[0114] For example, the second image is the image shown in c in FIG. 2 , and the third image is the image shown in d in FIG. 2 .

[0115] In some embodiments, the contouring parameters corresponding to the face shape of the portrait can be a contouring mask. The electronic device can generate a third image by performing an image operation on the contouring mask and the second image. In one example, the electronic device determines that each pixel in the contouring mask corresponds to a pixel on the face of the portrait in the second image, and performs an AND operation on each pixel in the contouring mask with the corresponding pixel in the second image to generate the third image.

[0116] The image processing method illustrated in the above embodiment identifies facial features of a person in an image captured by a camera, determines the person's face shape, obtains retouching parameters corresponding to the face shape, and then retouches the face based on the retouching parameters to enhance the portrait's beautification effect. This method allows for targeted facial retouching, with different retouching areas and degrees for different face shapes, meeting the personalized beautification needs of users with different face shapes.

[0117] In some embodiments, the electronic device may perform makeup processing on the facial parts of the person in the third image based on the features of the facial parts of the person to obtain a fourth image. The fourth image may be obtained by performing at least one of the following examples:

[0118] In one example, the electronic device obtains makeup parameters corresponding to the eye features of the portrait, such as eye bags parameters, eye light parameters, eye highlight parameters, eye shadow parameters, eyelash parameters, etc., based on the makeup parameters corresponding to the eye features of the portrait, and performs beauty makeup on the eyes of the portrait in the third image.

[0119] In one example, the electronic device obtains makeup parameters corresponding to the eyebrow features of the portrait, such as eyebrow dyeing parameters, based on the eyebrow features of the portrait, and performs beauty makeup processing on the eyebrows of the portrait in the third image based on the makeup parameters corresponding to the eyebrow features of the portrait.

[0120] In one example, the electronic device obtains makeup parameters corresponding to the lip features of the portrait, such as red lip parameters, based on the lip features of the portrait, and performs beauty makeup processing on the lips of the portrait in the third image based on the makeup parameters corresponding to the lip features of the portrait.

[0121] In one example, the electronic device obtains makeup parameters corresponding to the facial features of the portrait, such as blush parameters, based on the facial features of the portrait, and performs beauty makeup processing on the face of the portrait in the third image based on the makeup parameters corresponding to the facial features of the portrait.

[0122] The image processing method shown in the above embodiment recognizes the features of various parts of the face of the portrait and performs targeted beauty processing on each part of the face to achieve a better beauty effect.

[0123] The above-mentioned electronic devices may also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. The electronic devices may be mobile phones with shooting and display functions, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technologies and specific device forms adopted by the electronic devices.

[0124] For example, Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 9, the electronic device 100 includes: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor 180, a button 190, a camera 193, and a display 194.

[0125] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In some embodiments, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0126] It is understood that the interface connection relationship between the modules shown in the embodiment is only for illustrative purposes and does not limit the structure of the electronic device 100. In some embodiments, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0127] The processor 110 may include one or more processing units. The different processing units may be independent devices or integrated into one or more processors. The processor 110 may also be provided with a memory for storing instructions and data.

[0128] USB port 130 is an interface that complies with USB standards and may be a Mini USB port, a Micro USB port, a USB Type-C port, or the like. USB port 130 can be used to connect a charger to charge an electronic device, transfer data between the electronic device and peripheral devices, or connect headphones to play audio.

[0129] The charging management module 140 is configured to receive charging input from a charger. The power management module 141 is configured to connect the battery 142 , the charging management module 140 and the processor 110 .

[0130] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor. The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The wireless communication module 160 can provide solutions for wireless communications such as wireless local area networks (WLAN), Bluetooth, global navigation satellite systems (GNSS), frequency modulation (FM), NFC, and infrared technology (IR) applied to the electronic device 100.

[0131] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute instructions to generate or modify display information.

[0132] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0133] The electronic device 100 can implement a shooting function through an image signal processing (ISP) module, one or more cameras 193, a video codec, a GPU, one or more display screens 194, and an application processor.

[0134] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. Camera 193 includes a lens, an image sensor (such as a complementary metal oxide semiconductor image sensor (CIS)), a motor, and the like.

[0135] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, data files such as music, photos, and videos can be stored on the external memory card.

[0136] The internal memory 121 may be used to store one or more computer programs, which include instructions. The processor 110 may execute the instructions stored in the internal memory 121 to enable the electronic device 100 to perform various functional applications and data processing.

[0137] The sensor 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor 180K, an ambient light sensor, a bone conduction sensor, and the like.

[0138] The touch sensor 180K, also known as a touch panel, can be mounted on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a touch screen. The touch sensor 180K detects touch operations on or near the touch sensor and transmits the detected touch operations to the application processor to determine the type of touch event.

[0139] Keys 190 include a power button, a volume button, and the like. Keys 190 can be mechanical or touch-sensitive. Electronic device 100 can receive key inputs and generate key signal inputs related to user settings and function control of electronic device 100. For example, when the camera application is enabled, the user can trigger the camera to take photos or record videos by pressing the power button.

[0140] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of the present application, the software system of the layered architecture is an Android system as an example to illustrate the software structure of the electronic device.

[0141] Figure 10 is a schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application. A layered architecture divides the software system of an electronic device into several layers, each with distinct roles and divisions of labor. Layers communicate with each other via software interfaces. As shown in Figure 10, the electronic device includes an application layer, an application framework layer, a hardware abstraction layer, a driver layer, and a system service layer.

[0142] The application layer includes the camera app and third-party apps. The camera app is a system app, while third-party apps include but are not limited to short video apps, camera apps, and image processing apps. Users can use the camera app to capture images or videos, or use third-party apps to access camera-captured images or videos. In some embodiments, the application package may also include apps such as gallery, calendar, call, map, navigation, Bluetooth, music, video, and short messaging.

[0143] The application framework layer can provide an application programming interface (API) and a programming framework for the application programs in the application layer. In an embodiment of the present application, the application framework layer includes a camera management module and a window management module. The camera management module is responsible for managing the camera device information, and the camera application can obtain the camera characteristics, such as the number of cameras, shooting capabilities and other parameters, through the camera management module. In some embodiments, the camera management module can also be used to transmit data between the camera application and the camera hardware abstraction layer. For example, the camera application transmits a notification message to turn on the beauty function to the camera hardware abstraction layer through the camera management module, so that the camera hardware abstraction layer can call the camera algorithm module after receiving the original image taken by the camera and perform beauty processing on the portrait in the original image. The window management module is responsible for managing the windows in the application and the interaction with the user interface. For example, it is responsible for managing the camera application window and sending the window content (including the original image taken by the camera or the image after beauty processing) to the display driver for display.

[0144] The hardware abstraction layer is an interface layer located between the kernel layer and the hardware circuit. In an embodiment of the present application, the hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm module. The camera hardware abstraction layer can call the camera algorithm module to optimize the image or video taken by the camera. In some embodiments, the camera algorithm module includes a first image processing module and a second image processing module. The first image processing module is used to detect the image captured by the camera, obtain the facial feature information of the portrait in the image, such as the key point information of the portrait face, and determine the deformation parameters of each part of the face based on the facial feature information of the portrait, adjust the facial structure of the user in the image based on the deformation parameters, and transmit the image with the adjusted facial structure to the second image processing module. The second image processing module is used to obtain the facial feature information of the portrait in the image from the first image processing module, determine the face shape based on the facial feature information of the portrait, match the corresponding retouching parameters, and perform retouching on the portrait face based on the retouching parameters to obtain a beauty-processed image. The first image processing module and the second image processing module are processed in parallel to improve the image processing speed.

[0145] It should be noted that the first and second image processing modules are not limited to the hardware abstraction layer. In some embodiments, the first and second image processing modules may also be located in the application layer. For example, the first and second image processing modules may be integrated into a camera application or a third-party application, or the camera application or third-party application may implement a beauty function by calling the first and second image processing modules in the application layer.

[0146] In some embodiments, the first image processing module and the second image processing module may also be integrated into one image processing module, which has the functions of the first image processing module and the second image processing module.

[0147] The driver layer provides drivers for various hardware devices. In embodiments of the present application, the driver layer may include a camera driver and a display driver. The camera driver can be used to drive the camera of the electronic device to capture raw images. The display driver is used to drive the display screen of the electronic device to display the raw images captured by the camera or images that have been beautified.

[0148] Based on the electronic device hardware and software architecture shown above, the internal execution process of the device of the image processing method provided in the embodiment of the present application is described below in conjunction with a specific embodiment.

[0149] For example, FIG11 is a flow chart of the third embodiment of the image processing method provided by the present application. FIG11 takes the case where the portrait shooting mode is turned on and the camera application does not automatically turn on the beauty function as an example to illustrate the solution. As shown in FIG11, the image processing method may include the following steps:

[0150] S1101. In response to an operation of turning on the portrait shooting mode, the camera application sends a notification message of the portrait shooting mode to the camera management module.

[0151] For example, as shown in Figure 1, in response to the user selecting the portrait shooting mode in the shooting mode selection area 105 of the shooting interface 101, the camera application of the application layer sends a notification message of the portrait shooting mode to the camera management module of the application framework layer, and the camera management module can query the camera shooting parameters of the portrait shooting mode.

[0152] S1102. The camera management module sends control information of the portrait shooting mode to the camera driver through the camera hardware abstraction layer.

[0153] The control information of the portrait shooting mode may include camera shooting parameters of the portrait shooting mode, such as the identification of the camera corresponding to the portrait shooting mode and the focal length value of the camera, etc. The camera corresponding to the portrait shooting mode includes a telephoto camera.

[0154] S1103. The camera driver drives the corresponding camera to operate and obtain a first image from the camera.

[0155] The camera driver drives the corresponding camera to work based on the camera shooting parameters in the control information of the portrait shooting mode. The first image is an original image shot by the camera, and the original image can be understood as an unprocessed image.

[0156] S1104. The camera driver sends the first image to the camera hardware abstraction layer.

[0157] If the camera application does not have the beauty feature enabled, upon receiving the first image sent by the camera driver, the camera hardware abstraction layer will not call the first image processing module or the second image processing module, that is, it will not perform image processing (e.g., beauty processing) on ​​the first image. The camera hardware abstraction layer directly transmits the first image to the camera application via the camera management module, so that the camera application, via the window management module, sends the first image to the display driver for display. This process is the normal process for portrait shooting mode, and the preview interface of the camera application displays the original image captured by the camera.

[0158] In some embodiments, the image processing method further includes:

[0159] S1105. In response to the operation of turning on the beauty function, the camera application sends a notification message to the camera management module to turn on the beauty function. For example, as shown in FIG1 , in response to the user clicking the beauty function switch 103 in the preview area 102 of the shooting interface 101 , the camera application sends a notification message to the camera management module to turn on the beauty function.

[0160] S1106. The camera management module sends a notification message to the camera hardware abstraction layer to enable the beauty function.

[0161] After receiving the first image sent by the camera driver ( S1104 ), the camera hardware abstraction layer executes S1107 .

[0162] S1107. The camera hardware abstraction layer sends an image processing request to the first image processing module, the image processing request including the first image, and triggering the first image processing module to perform facial structure adjustment on the first image.

[0163] S1108. The first image processing module identifies facial feature information in the first image.

[0164] The first image processing module may be pre-installed with a facial recognition model and a facial key point detection model. The first image processing module uses the facial recognition model to identify whether the first image contains a human face. If the first image is found to contain a human face, the first image processing module further uses the facial key point detection model to identify key points of various parts of the human face to obtain facial feature information of the human face in the first image, the facial feature information including key point information of the human face.

[0165] In some embodiments, the facial feature information may further include facial contour information, and the facial contour information may be determined based on facial key points.

[0166] S1109. The first image processing module determines the deformation parameters of each facial part based on the facial feature information.

[0167] S1110: The first image processing module optimizes the facial structure based on the deformation parameters to obtain a second image. The second image is an image obtained by optimizing the facial structure of the portrait in the first image.

[0168] In this embodiment, S1109 and S1110 can refer to S304 and S305 of the previous embodiment respectively, and their implementation principles and effects are similar, which will not be elaborated here.

[0169] S1111. The first image processing module sends the second image to the second image processing module.

[0170] After S1108, it also includes:

[0171] S1112. The first image processing module sends facial feature information to the second image processing module.

[0172] S1113. The second image processing module determines the face shape based on the facial feature information.

[0173] S1114. The second image processing module obtains the contouring parameters corresponding to the face shape from the database.

[0174] S1115. The second image processing module performs face retouching on the second image based on the face retouching parameters to obtain a third image. The third image is an image after the face retouching is performed on the portrait in the second image.

[0175] In this embodiment, S1113 to S1115 can refer to S601 to S603 of the previous embodiment respectively, and their implementation principles and effects are similar, which will not be elaborated here.

[0176] S1116. The second image processing module sends the third image to the camera application.

[0177] The second image processing module sends the third image to the camera hardware abstraction layer, the camera hardware abstraction layer sends the third image to the camera management module, and the camera management module sends the third image to the camera application.

[0178] S1117. The camera application sends the third image to the display driver to display the third image.

[0179] The camera application sends the third image to the window management module, which then sends the third image to the display driver for display. For example, as shown in FIG1(b), the beautified image, i.e., the third image, is displayed in the preview area 108 of the camera application's shooting interface.

[0180] The image processing method of this embodiment shows the interaction of various modules inside the electronic device when the beauty function is turned on. The camera application calls the underlying image processing modules of the electronic device, namely the first image processing module and the second image processing module, to optimize and reshape the facial structure of the original image portrait, and presents the beautified portrait in the preview screen to meet the personalized beauty needs of different users.

[0181] In some embodiments, when the portrait shooting mode is turned on, the camera application may also automatically turn on the beauty function. After receiving the notification message of the portrait shooting mode from the camera application, the camera management module may query whether the automatic turning on of the beauty function is configured in the portrait shooting mode. If the automatic turning on of the beauty function is configured in the portrait shooting mode, the control information of the portrait shooting mode may also include a notification message for turning on the beauty function, so that the camera hardware abstraction layer calls the first image processing module and the second image processing module to perform portrait detection and beauty processing (the aforementioned S1107 to S1115).

[0182] Based on the aforementioned embodiments, an embodiment of the present application proposes an image processing method for use in an electronic device, the method comprising: in response to an operation to activate the beauty function of a camera application, obtaining a first image including a portrait face captured by the camera; and displaying a third image on the shooting interface of the camera application, the third image being an image obtained by superimposing a retouching mask image on the first image, the retouching mask image being a mask image that matches the face shape of the portrait face, and the retouching mask image being used to indicate the position, shape, and degree of highlight and shadow areas of the portrait face. The first image is the original image captured by the camera, and the above method takes into account the differences in face shapes of different portraits, and superimposes a retouching mask image that matches the face shape of the portrait face on the original image to meet the personalized beauty needs of different users and enhance the beauty effect of the portrait.

[0183] In an optional embodiment, before the third image is displayed on the camera application's capture interface, the method further includes: obtaining facial feature information of the person in the first image; determining the person's facial shape based on the facial feature information; obtaining a retouching mask image that matches the person's facial shape from a database, the database including retouching mask images corresponding to different facial shapes; and superimposing the first image and the retouching mask image to obtain the third image. The above method enhances the facial contours of the person in the image by obtaining facial features of the person to determine the person's facial shape and obtaining a retouching mask image corresponding to the person's facial shape.

[0184] In one optional embodiment, obtaining facial feature information of a person's face in a first image includes: determining the image region in the first image where the person's face is located; and identifying key points of various facial features of the person in the first image using a preset facial key point detection model to obtain key point information of the person in the first image. The key point information includes the positional information of the key points of various facial features. The above method obtains key point information of various facial features of the person through detection using the facial key point detection model, providing data support for face shape analysis, matching, and facial adjustment.

[0185] In an optional embodiment, the face shape of the portrait face is determined based on the feature information of the portrait face, including: determining the facial contour information of the portrait based on the key point information of the portrait face in the first image; determining the face shape of the portrait face based on the facial contour information; or, determining the face shape similarity and facial feature similarity of the portrait face in the first image with multiple standard faces through a preset portrait similarity model, determining the total similarity between the portrait face and the multiple standard faces, and taking the face shape of the standard face with the largest total similarity as the face shape of the portrait face in the first image. The above-mentioned first method for determining the face shape of a portrait is to determine the facial contour through the key points of the portrait, and match the corresponding face shape based on the facial contour. The above-mentioned second method for determining the face shape of a portrait is to calculate the similarity between the face shape of the portrait and different standard face shapes based on the portrait face similarity model, and then determine the face shape of the portrait.

[0186] In an optional embodiment, the positions of the highlight area and the shadow area of ​​the portrait face in the retouching mask images corresponding to different face shapes are different, and the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask images corresponding to different face shapes are different.

[0187] In an optional embodiment, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different, including: if the face shape of the portrait face is an oval face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are water drop shapes; or, if the face shape of the portrait face is a round face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are lightning shapes; or, if the face shape of the portrait face is a rectangular face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are comb shapes; or, if the face shape of the portrait face is a diamond face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are semi-arc shapes; or, if the face shape of the portrait face is an elliptical face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are Z shapes; or, if the face shape of the portrait face is a square face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are L shapes.

[0188] The above two embodiments show the differences in the retouching mask images corresponding to different face shapes. After determining the face shape of the portrait, by matching the retouching mask image corresponding to the face shape of the portrait, differentiated retouching of the portrait face can be achieved, thereby improving the portrait beauty effect.

[0189] In an optional embodiment, the method further includes: determining the deformation parameters of various parts of the face based on the characteristic information of the portrait face; adjusting the structure of the portrait face in the first image by region based on the deformation parameters of various parts of the face to obtain a second image; and superimposing the first image and the retouching mask image to obtain a third image, including: superimposing the second image and the retouching mask image to obtain a third image. The above method first optimizes the facial structure of the portrait, and then, based on the optimized facial structure, retouches the portrait face to enhance the portrait beauty effect. It is worth noting that when optimizing the facial structure of the portrait, the structure of various parts of the portrait face is adjusted by region to avoid affecting adjacent parts.

[0190] In an optional embodiment, the deformation parameters of each facial part are determined based on the characteristic information of the portrait face, including: based on the characteristic information of the portrait face and the facial characteristic information of the standard face, the structure of the portrait face in the first image is adaptively adjusted within a preset numerical range to determine the deformation parameters of each facial part. The deformation parameters of each facial part should be set within a reasonable numerical range, that is, the deformation parameters of each part should be set within the preset numerical range corresponding to each part. It can be understood that if the deformation parameters are too large, facial distortion may occur, and if the deformation parameters are too small, the facial structure optimization is not obvious. Reasonable facial deformation parameters are generated by adaptive parameter adjustment to optimize the facial structure of the portrait.

[0191] In an optional embodiment, the standard face includes a plurality of standard faces of different face shapes; based on the feature information of the portrait face and the facial feature information of the standard face, the structure of the portrait face in the first image is adaptively adjusted within a preset numerical range to determine the deformation parameters of various parts of the portrait face, including: determining the face shape of the portrait face based on the feature information of the portrait face; obtaining the facial feature information of the standard face corresponding to the face shape of the portrait face; based on the feature information of the portrait face and the facial feature information of the standard face corresponding to the face shape of the portrait face, adaptively adjusting the structure of the portrait face in the first image within a preset numerical range to determine the deformation parameters of various parts of the portrait face. The above method combines the facial features of the standard face corresponding to the face shape of the portrait face to adaptively adjust the parameters of various parts of the portrait face to generate reasonable facial deformation parameters and optimize the face structure of the portrait.

[0192] In an optional embodiment, the method further includes: in response to a first operation of enabling a portrait shooting mode of a camera application, enabling a beauty function. Alternatively, in response to the first operation of enabling the portrait shooting mode, a first control is displayed on the shooting interface, the first control being used to toggle the beauty function on or off; and in response to a second operation on the first control, enabling the beauty function.

[0193] Exemplarily, with reference to FIG1 , the first operation may be an operation in which a user selects a portrait shooting mode in the shooting mode selection area 105 of the camera application shooting interface 101. In one example, in response to the first operation, the electronic device directly turns on the beauty function. In another example, in response to the first operation, the camera application switches to the portrait shooting mode, and the beauty function is not turned on at this time. Continuing to refer to FIG1 , the first control may be the beauty function switch 103 in the preview area 102 of the camera application shooting interface 101, and the second operation may be an operation of clicking the beauty function switch 103. In response to the second operation, the electronic device turns on the beauty function. Unlike the previous example, the beauty function needs to be turned on by the user.

[0194] The above methods illustrate two methods for turning on the beauty function. After turning on the beauty function, the electronic device can execute the above image processing method to meet the personalized beauty needs of different users.

[0195] It should be noted that the embodiments of the present application do not particularly limit the specific structure of the execution subject of an image processing method. As long as the code storing the image processing method of the embodiments of the present application can be executed to perform processing according to the image processing method provided by the embodiments of the present application, it is sufficient. For example, the execution subject of an image processing method provided by the embodiments of the present application can be a functional module in an electronic device that can call and execute a program, or a processing device used in an electronic device, such as a chip.

[0196] In the above embodiments, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functions.

[0197] Therefore, the modules of each example described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0198] An embodiment of the present application provides an electronic device, including: a memory and a processor, wherein the processor is used to call a computer program in the memory to execute a technical solution as in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar to those of the aforementioned related embodiments and will not be repeated here.

[0199] The memory may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0200] The memory can be independent and connected to the processor via a communication line, or it can be integrated with the processor.

[0201] The processor may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0202] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the technical solution of any of the above embodiments. Its implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be repeated here.

[0203] An embodiment of the present application provides a chip, which includes a processor. The processor is used to call a computer program in a memory to execute the technical solution in any of the above embodiments. Its implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be repeated here.

[0204] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the technical solution in any of the above embodiments. The implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be repeated here.

[0205] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: Applied to electronic equipment, the method includes: In response to an operation of turning on a beauty function of a camera application, obtaining a first image including a portrait face captured by a camera; A third image is displayed on the shooting interface of the camera application. The third image is an image obtained by superimposing a retouching mask image on the first image. The retouching mask image is a mask image that matches the facial shape of the portrait face. The retouching mask image is used to indicate the position, shape and degree of the highlight area and shadow area of ​​the portrait face.

2. The method according to claim 1, characterized in that Before the shooting interface of the camera application displays the third image, the method further includes: Acquire feature information of the face of the person in the first image; Determining the face shape of the portrait face based on the feature information of the portrait face; Obtaining a retouching mask image that matches the face shape of the portrait face from a database, wherein the database includes retouching mask images corresponding to different face shapes; The first image and the retouching mask image are superimposed to obtain the third image.

3. The method according to claim 2, characterized in that Acquiring facial feature information of the portrait in the first image includes: Determining an image region in the first image where the face of the portrait is located; The key points of various parts of the face of the person in the first image are identified by using a preset facial key point detection model to obtain key point information of the face of the person in the first image.

4. The method according to claim 2 or 3, characterized in that Determining the face shape of the portrait face based on feature information of the portrait face includes: Determining facial contour information of the portrait based on key point information of the face of the portrait in the first image; determining a face shape of the portrait based on the facial contour information; or Through a preset portrait similarity model, the facial shape similarity and facial feature similarity between the portrait face in the first image and multiple standard faces are determined, the total similarity between the portrait face and the multiple standard faces is determined, and the face shape of the standard face with the largest total similarity is used as the face shape of the portrait face in the first image.

5. The method according to any one of claims 1 to 4, characterized in that The positions of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different, and the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different.

6. The method according to claim 5, characterized in that The shapes of the highlight area and shadow area of ​​the portrait face in the retouching mask image corresponding to different face shapes are different, including: If the face shape of the portrait face is an oval face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are teardrop shapes; or If the face shape of the portrait face is a round face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are lightning shapes; or If the face shape of the portrait face is a rectangular face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are comb-shaped; or If the face shape of the portrait face is a diamond face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are semi-arc shapes; or If the face shape of the portrait face is an elliptical face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are Z-shaped; or If the face shape of the portrait face is a square face, the shapes of the highlight area and the shadow area of ​​the portrait face in the retouching mask image are L-shaped.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determining deformation parameters of various parts of the face based on feature information of the portrait face; Based on the deformation parameters of each part of the face, adjusting the structure of the face of the portrait in the first image by region to obtain a second image; The first image and the retouching mask image are superimposed to obtain the third image, including: The second image and the retouching mask image are superimposed to obtain the third image.

8. The method according to claim 7, characterized in that Determining deformation parameters of various facial parts based on the facial feature information of the portrait includes: Based on the facial feature information of the portrait face and the facial feature information of the standard face, the structure of the portrait face in the first image is adaptively adjusted within a preset numerical range to determine the deformation parameters of various parts of the portrait face.

9. The method according to claim 8, characterized in that The standard face includes a plurality of standard faces of different face shapes; based on the feature information of the portrait face and the facial feature information of the standard face, adaptively adjusting the structure of the portrait face in the first image within a preset value range to determine deformation parameters of various parts of the portrait face, including: Determining the face shape of the portrait face based on the feature information of the portrait face; Obtaining facial feature information of a standard face corresponding to the face shape of the portrait face; Based on the facial feature information of the portrait and the facial feature information of a standard face corresponding to the face shape of the portrait, the structure of the portrait in the first image is adaptively adjusted within the preset numerical range to determine the deformation parameters of each part of the portrait.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: In response to a first operation of starting a portrait shooting mode of a camera application, turning on the beautification function; or In response to a first operation of turning on the portrait shooting mode, the shooting interface displays a first control, which is used to trigger turning on or off the beauty function; in response to a second operation acting on the first control, the beauty function is turned on.

11. An electronic device, characterized in that: The electronic device comprises: a memory and a processor, wherein the processor is configured to call a computer program in the memory to execute the method according to any one of claims 1 to 10.

12. A chip, characterized in that: The chip includes a processor, and the processor is configured to call a computer program in a memory to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 10.

Images