Image processing method, device and electronic device

Abstract

The present application discloses an image processing method, device and electronic device, which belongs to the field of communication technology and can solve the problem that the processing process of the related technology cannot effectively beautify the user's facial photos at other angles except the frontal facial photos, resulting in the user being unable to obtain beauty photos at any angle. The method includes: the electronic device obtains the feature point information of the jaw feature points among the facial feature points of the user; the electronic device calculates the maxillofacial parameters of the user based on the feature point information of the jaw feature points; the electronic device determines the face shape of the user based on the maxillofacial parameters; the electronic device adjusts the maxillofacial parameters based on the preset maxillofacial parameters that match the face shape of the user.

Description

Technical Field

[0001] The present application belongs to the field of technical communications, and specifically relates to an image processing method, device and electronic equipment. Background Art

[0002] With the development of electronic equipment technology, electronic equipment has more and more camera functions, among which the beauty function in the camera function is favored by more and more users. In the related technology, after the electronic equipment collects the front face photo of the user through the camera technology, the facial features and face information of the user in the photo can be compared with the standard face shape pre-stored in the electronic equipment, and then the angle or proportion of the facial features and face shape can be adjusted.

[0003] However, the above processing cannot effectively beautify the user's facial photos taken at other angles (face photos taken at the side or close to the side) except the frontal facial photos. Therefore, the user cannot obtain beauty photos taken at any angle. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method, device and electronic device, which can solve the problem that the processing process of the related technology cannot effectively beautify the user's facial photos at other angles except the front facial photos, resulting in the user being unable to obtain beauty photos at any angle.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which includes: an electronic device obtains feature point information of a jaw feature point among facial feature points of a user; the electronic device calculates the maxillofacial parameters of the user based on the feature point information of the jaw feature point; the electronic device determines the face shape of the user based on the maxillofacial parameters; the electronic device adjusts the maxillofacial parameters based on preset maxillofacial parameters that match the face shape of the user.

[0007] In a second aspect, an embodiment of the present application provides an image processing device, comprising an acquisition module, a calculation module, a determination module and an adjustment module; the acquisition module is used to acquire feature point information of the jaw feature points among the facial feature points of the user; the calculation module is used to calculate the maxillofacial parameters of the user based on the feature point information of the jaw feature points; the determination module is used to determine the face shape of the user based on the maxillofacial parameters; the adjustment module is used to adjust the maxillofacial parameters based on preset maxillofacial parameters that match the face shape of the user.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In the embodiment of the present application, after obtaining the feature point information of the jaw feature point in the user's facial feature points, the electronic device can calculate the user's maxillofacial parameters based on the feature point information of the jaw feature point, and determine the user's face shape based on the maxillofacial parameters, and finally determine the preset maxillofacial parameters that match the user's face shape, and then adjust the maxillofacial parameters. In this way, by obtaining the jaw feature of the user, the parameters of the user's maxillofacial part can be adjusted in a targeted manner, and then provide accurate and effective correction parameters for the electronic device when processing facial photos of the user at angles other than the front (for example, facial photos of the user at the side or other angles), so that the electronic device can effectively process facial photos of the user at angles other than the front. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram of an E line applied by an image processing method provided in an embodiment of the present application;

[0013] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0014] Figure 3 is one of the face shape schematic diagrams used in an image processing method provided in an embodiment of the present application;

[0015] Figure 4 is one of the face shape schematic diagrams used in an image processing method provided in an embodiment of the present application;

[0016] Figure 5 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0017] Figure 6 One of the structural schematic diagrams of an electronic device provided in an embodiment of the present application;

[0018] Figure 7 The second structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0021] The following explanations are given for the nouns appearing in the embodiments of the present application:

[0022] 1. E line

[0023] The E line, also known as the aesthetic standard line, is a line that passes through the soft tissue premental point and the nasal tip point. In a well-balanced profile, the lower lip is on the E line and the upper lip is on the line or 0.5-1 mm behind the line.

[0024] like Figure 1 As shown, the E line 20 is formed by the line connecting the nose tip point 21 and the soft tissue pre-chin point 22.

[0025] 2. Mandibular part

[0026] The mandible mainly consists of three parts: the chin, the mandibular margin and the mandibular angle. The chin is composed of the connection of the horizontal branches of the mandible on both sides and is located below the midline of the face. The chin-neck angle formed by the lower edge of the chin and the upper part of the neck is the dividing line between the face and the neck. The chin, the zygomatic bones on both sides and the mandibular angle area constitute the basic contour of the face, and together with the tissues and organs such as the mid-face, nose and lips, constitute the basic characteristics of the facial morphology.

[0027] Among them, the shape, size and position of the chin have a great impact on facial appearance. Underdeveloped chin can lead to a "pointed mouth", "pointed face" or "bird-shaped face". Overdeveloped chin can cause a "long face" or "horse face". Deviated chin can cause asymmetry of the entire face.

[0028] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0029] The image processing method provided in the embodiment of the present application can be applied to the scenario where the user takes a portrait.

[0030] For the scenario where the user takes a portrait, suppose the user uses an electronic device to take a photo of his or her side face. After taking the photo, an unprocessed photo 1 is obtained. At this time, the electronic device uses the beauty function to process photo 1. Because in the related art, the beauty function mainly adjusts the face shape, facial features and skin properties (for example, skin color, skin fineness) of the user's front face, therefore, for a photo such as photo 1 that only shows the side face but does not show the complete front facial information, the electronic device cannot make effective adjustments, for example, adjust the user's facial bones, side face shape, etc. In general, after the electronic device uses the beauty function to adjust photo 1, the photo 2 actually presented is basically unadjusted relative to photo 1. Obviously, this basically ineffective adjustment does not meet the user's beauty needs.

[0031] In the embodiment of the present application, after acquiring photo 1, the electronic device can obtain the feature point information of the jaw feature point among the facial feature points of the user in photo 1, calculate the maxillofacial parameters of the user according to the feature point information of the jaw feature point, determine the face shape of the user according to the jaw feature parameters, and finally determine the preset maxillofacial parameters that match the face shape of the user, and then adjust the jaw facial parameters, and finally the adjusted photo 2 effectively adjusts the maxillofacial part of the user relative to photo 1. In this way, by acquiring the jaw feature of the user in photo 2, the parameters of the maxillofacial part of the user can be adjusted in a targeted manner, and then provide accurate and effective correction parameters for the electronic device when processing facial photos of the user at angles other than the front (for example, facial photos of the user at the side or other angles), so that the electronic device can effectively process facial photos of the user at angles other than the front.

[0032] This embodiment provides an image processing method, such as Figure 2 As shown, the image processing method includes the following steps 301 to 304:

[0033] Step 301: The image processing apparatus obtains feature point information of the jaw feature points among the facial feature points of the user.

[0034] In an embodiment of the present application, the facial feature points of the user may include all feature points of all directions of the user's face. In one example, the facial feature points may include front facial feature points and front facial shape feature points, and may also include side facial feature points and side facial shape feature points.

[0035] In the embodiment of the present application, the image processing device can obtain the facial feature points of the user through full face scanning technology. For example,

[0036] Exemplarily, the image processing device can scan the entire face of the user through the 3D face scanning technology in the electronic device to obtain three-dimensional data information of the entire facial image, which may include a frontal facial image and facial images from other angles. Then, the user's face 3D model is generated by combining the data information of these facial images. The electronic device can set the key points on the user's face 3D model as facial feature points, and each facial feature point contains feature point information. Among them, when the above-mentioned electronic device searches for the key points on the user's face 3D model, it can search for the key points in the user's face 3D model according to the key point distribution model on the preset 3D model pre-stored in the electronic device.

[0037] Among them, in the process of 3D face scanning technology in electronic devices scanning the entire face of the user and obtaining the three-dimensional data information of the entire facial image, the three-dimensional data of the face can be obtained through the combination of infrared lens, floodlight sensor group and dot matrix projector. Among them, the infrared lens captures the facial image information of the person to generate three-dimensional data, and the floodlight sensor group cooperates with the infrared lens to adapt to various light environments so that it is not affected by day and night. After the facial image information is collected, the dot matrix projector projects countless light spots on the person's face for 3D stereo modeling.

[0038] In the embodiment of the present application, the above-mentioned jaw feature points may be jaw feature points selected by the image processing device from facial feature points.

[0039] Exemplarily, multiple sets of image relationship models between the mandibular part and the face may be pre-stored in the electronic device. In one example, for different face shapes, the relationship between the mandibular part and the overall face is different (for example, the area of ​​the mandibular part occupied by the overall face, the relationship between the mandibular part and the overall face). Therefore, the electronic device may pre-store multiple sets of relationships between the mandibular part and the overall face according to different face shapes, so that the image processing device can filter out the mandibular part feature points from the user's facial feature points and perform further processing.

[0040] In one example, the feature points of the mandibular part may include the highest point of the nose tip, the highest point of the lips (ie, the location of the lip bead), and the highest point of the chin.

[0041] In the embodiment of the present application, the above-mentioned feature point information may be data information of the feature point, for example, coordinate information of the feature point.

[0042] Step 302: The image processing device calculates the maxillofacial parameters of the user according to the feature point information of the feature points of the mandibular part.

[0043] In the embodiment of the present application, the above-mentioned maxillofacial parameters may be parameter information corresponding to the maxillofacial feature points.

[0044] In one example, the above-mentioned maxillofacial parameters may be shape parameters, angle parameters, area parameters of the user's side face, and proportion parameters compared with the user's facial feature points, for example, the area ratio of the maxillofacial face compared with the user's overall face.

[0045] Step 303: The image processing device determines the face shape of the user according to the maxillofacial parameters.

[0046] In an embodiment of the present application, the face shape of the user can be a face shape model pre-stored in the electronic device by the image processing device, and the face shape model closest to the user's face shape can be selected as the user's face shape. The face shape model pre-stored in the electronic device can be the most common face shape model in existing portraits.

[0047] In one example, the above face shape model can be divided into three types: straight face type, concave face type and convex face type.

[0048] The first type: straight face. The above straight face type generally meets the aesthetic E line, with the nose tip, lips, and chin basically in a straight line; it is also a beauty that is generally accepted by the public, and the face looks dignified and generous.

[0049] The second type: convex face. The convex face type actually presents the image of the lower jaw part with protruding lips, which is generally caused by what ordinary people call "buck teeth". Some people have buck teeth on the upper teeth; some people have buck teeth on both the upper and lower teeth. People with convex faces are usually considered not to be particularly intellectual and fashionable.

[0050] The third type: concave face. The above concave face type actually presents the image of the lower jaw part with a protruding chin. Generally, people call it "shoe horn face", and the more typical face is "underbite". During the development of the upper and lower jaws, some people have normal upper jaw development and protruding chin; some people have normal chin development but insufficient upper jaw development; and there is also a type of person with insufficient upper jaw development and a protruding chin. All three situations make the middle part of the face look sunken. Usually give people a feeling of being more cold and serious.

[0051] Step 304: The image processing device adjusts the maxillofacial parameters based on preset maxillofacial parameters that match the face shape of the user.

[0052] Optionally, the preset maxillofacial parameters include parameters corresponding to feature point information of feature points of the mandibular part of the preset maxillofacial region.

[0053] Exemplarily, the above-mentioned preset maxillofacial parameters may be pre-stored in the electronic device, or may be user-defined, which is not limited in the embodiments of the present application.

[0054] Exemplarily, the preset maxillofacial parameters may be the maxillofacial parameters of the most common face shape in existing portraits.

[0055] In one example, the electronic device may pre-store the maxillofacial parameters of the above three face models as preset maxillofacial parameters, that is, the electronic device may pre-store the straight face type maxillofacial parameters, convex face type maxillofacial parameters and concave face type maxillofacial parameters as preset maxillofacial parameters.

[0056] In the embodiment of the present application, the image processing device can adjust the shape parameters, angle parameters, area parameters and proportion parameters of the user's maxillofacial parameters compared with the user's facial feature points.

[0057] It should be noted that when adjusting the user's jaw-face parameters, the image processing device can perform 3D stereoscopic adjustments on all feature points of the user's jaw, rather than just adjusting a certain local area or line. For example, when the user's face is convex, the image display device can make the maximum angle of inward adjustment to the protruding apex, and gradually reduce the adjustment angle from the most protruding part to the sides of the user's face. In this way, the final effect of the adjustment can be enhanced, making the adjusted image more realistic and three-dimensional in terms of perception.

[0058] In one example, when the user's face shape is a straight face type, the image processing device does not need to correct the user's maxillofacial parameters and keeps them as they are.

[0059] In one example, when the user's face shape is convex, the image processing device can use an algorithm to adjust the feature point information of the user's mouth so that the user's mouth is corrected inward, the mouth is retracted, and the protruding mouth is corrected. At the same time, since most users with convex faces have a retracted chin, the chin part can be slightly pulled out, resulting in the effect of a slightly tilted chin.

[0060] In one example, when the user's face shape is concave, the image processing device can use an algorithm to adjust the feature point information of the user's mouth so that the user's mouth is corrected outward, and the concave mouth becomes convex until it is on the same horizontal line as the nose tip and the chin vertex.

[0061] Example 1: Assume that a user with a convex face needs to use an electronic device to adjust the side face photo he or she took of himself or herself. The electronic device first scans the user's face through 3D scanning software, and obtains the coordinate information of the feature points of the lower jaw part of the user's facial feature points (i.e., the above-mentioned feature point information). According to the coordinate information of the feature points of the lower jaw part, the maxillofacial parameters of the user's maxillofacial face can be calculated, including the shape parameters, angle parameters, area parameters of the maxillofacial face, and the proportion parameters compared with the user's facial feature points. Then, based on the shape parameters, angle parameters, area parameters, and proportion parameters compared with the user's facial feature points of the above-mentioned user's maxillofacial face, the electronic device can determine that the user's face shape is a convex face. Finally, the electronic device can use the preset maxillofacial parameters that match the convex face shape to adjust the maxillofacial parameters in the above-mentioned side face photo, and appropriately reduce the angle parameters of the protruding part in the user's side face photo with reference to the preset maxillofacial parameters, so that the user with a convex face finally presents a standard face shape.

[0062] In the image processing method provided in the embodiment of the present application, after the image processing device obtains the feature point information of the jaw feature point in the user's facial feature points, it can calculate the user's maxillofacial parameters based on the feature point information of the jaw feature point, and determine the user's face shape based on the maxillofacial parameters, and finally determine the preset maxillofacial parameters that match the user's face shape, and then adjust the maxillofacial parameters. In this way, by obtaining the jaw feature of the user, the parameters of the user's maxillofacial part can be adjusted in a targeted manner, and then provide accurate and effective correction parameters for the electronic device when processing the user's facial photos at angles other than the front (for example, the user's facial photos at the side or other angles), so that the electronic device can effectively process the user's facial photos at angles other than the front.

[0063] Optionally, in the embodiment of the present application, in the above step 302, the image processing method provided by the embodiment of the present application may include the following steps A1 and A2:

[0064] Step A1: The image processing device identifies the E-line feature points from the above-mentioned jaw part feature points.

[0065] Step A2: The image processing device calculates the maxillofacial parameters of the user according to the feature point information of the E-line feature points.

[0066] Exemplarily, the image processing device may identify the E-line feature points according to the distribution of the feature points of the mandibular part and the preset relationship between the E-line feature points and the mandibular feature points. The preset relationship between the E-line feature points and the mandibular feature points may be the relative position relationship of the coordinates between the E-line feature points and the mandibular feature points.

[0067] In one example, the E-line feature point can be the highest point of the user's nose, mouth, and chin among the feature points of the mandibular part, or the middle point of the user's nose, mouth, and chin among the feature points of the mandibular part, or the most prominent feature point of the user's nose, mouth, and chin among the feature points of the mandibular part.

[0068] Example 2: In combination with Example 1 above, when the electronic device calculates the maxillofacial parameters of the user's jaw and face based on the coordinate information of the feature points of the mandibular part, the feature point information of the E-line feature points can be filtered out based on the coordinate information of the feature points of the mandibular part. Since the user takes a side face photo of himself, the electronic device can calculate the maxillofacial parameters of the user based on the coordinate information corresponding to the highest points of the nose, mouth and chin of the user in the side face photo after identifying them.

[0069] In this way, the image processing device only needs to obtain the E-line feature points among the feature points of the mandibular part to calculate the user's maxillofacial parameters, and then perform face shape analysis and facial image adjustment, which reduces the amount of calculation and saves the resources of the electronic equipment required for calculation, thereby saving power consumption.

[0070] Optionally, in the embodiment of the present application, the E-line feature points include the nose vertex, mouth vertex and chin vertex of the user's face. On this basis, before the above step A1, the image processing method provided in the embodiment of the present application may also include the following step B:

[0071] Step B: The image processing device determines the mouth state of the user according to the feature point information of the facial feature points of the user.

[0072] Exemplarily, the electronic device may pre-store feature point information of facial feature points corresponding to expressions and states corresponding to the facial features of the portrait.

[0073] In one example, the electronic device may pre-store feature point information of feature points corresponding to the mouth contour of a face image in two states: the first state, the mouth is open; the second state, the mouth is closed. The feature point information of the feature points corresponding to the mouth contour in the open mouth state is completely different from the feature point information of the feature points corresponding to the mouth contour in the open mouth state.

[0074] Based on the above step B, according to different states of the mouth, the above mouth vertices can be divided into the following two cases.

[0075] First case:

[0076] Exemplarily, when the above-mentioned mouth state is a tooth-exposed state, the above-mentioned mouth vertex is the highest point of the teeth that expose the teeth.

[0077] It can be understood that when the mouth is open, the feature point information of the feature points corresponding to the mouth contour obtained by the image display device successfully matches the pre-stored first state, and the mouth state is determined to be the tooth-exposed state, and the mouth image is displayed as the teeth exposed. In this state, the electronic device can obtain the feature point information corresponding to the feature points of the exposed teeth part, and after analysis, the highest point of the feature points of the exposed teeth part is used as the vertex of the mouth.

[0078] It should be noted that the highest point mentioned above may be the most prominent feature point of the user's mouth.

[0079] Furthermore, after obtaining the highest point of the tooth, when adjusting the maxillofacial parameters, the image processing device can adjust the maxillofacial parameters corresponding to the feature point information of the feature point of the exposed tooth part. That is, the image processing device can adjust the state of the exposed part of the tooth in the image by adjusting the maxillofacial parameters corresponding to the tooth part, for example, adjusting buck teeth to normal teeth.

[0080] Example 3: Assuming that the side profile photo taken by the user is a side profile photo with teeth exposed, then in combination with the above Examples 1 and 2, when the electronic device uses the preset maxillofacial parameters that match the convex face shape to adjust the maxillofacial parameters in the above side profile photo, the highest point of the teeth in the user's side profile photo can be used as the highest point of the mouth, connected in a straight line with the highest point of the nose and the highest point of the chin, and the angle parameters of the protruding part of the mouth can be appropriately adjusted to be reduced with reference to the preset maxillofacial parameters, so that the user with a convex face ultimately presents a standard face shape.

[0081] Second case:

[0082] Exemplarily, when the mouth state is a non-teeth-exposed state, the mouth apex is the highest point of the lips.

[0083] It is understandable that when the mouth is closed, the feature point information of the feature points corresponding to the mouth contour obtained by the image display device successfully matches the pre-stored second state, and the mouth state is determined to be a non-teeth-exposed state, and the mouth image is displayed as non-teeth-exposed. In this state, the electronic device can analyze the feature point information corresponding to the feature points of the lip contour to obtain the mouth feature vertex. Generally, when the mouth is closed, the mouth vertex is the lip bead point.

[0084] In this way, the electronic device determines the user's mouth state and adjusts the strategy for obtaining E-line feature points, thereby using E-line feature points that match the mouth state, and can effectively, specifically and accurately adjust the user's lower jaw.

[0085] Optionally, in the embodiment of the present application, in the above step 302, the image processing method provided by the embodiment of the present application may include the following step C:

[0086] Step C: The image processing device obtains the maxillofacial parameters of the user by calculating the angle between a first line connecting the nose vertex and the mouth vertex of the user's face and a second line connecting the mouth vertex and the chin vertex of the user's face.

[0087] Exemplarily, the above-mentioned nose vertex, mouth vertex and chin fixed point can refer to the above description and will not be repeated here.

[0088] Exemplarily, by calculating the angle between the first connecting line and the second connecting line, the user's maxillofacial parameters are obtained, and then the user's face shape can be determined based on the maxillofacial parameters, for example, a straight face shape, a concave face shape, or a convex face shape.

[0089] It should be noted that the above-mentioned angle is the angle between the above-mentioned first connecting line and the above-mentioned second connecting line facing the same direction. In one example, the above-mentioned angle may be an angle facing the outside of the face, and in another example, the above-mentioned angle may be an angle facing the inside of the face.

[0090] In this way, the image processing device only needs to calculate the angle between the first line connecting the nose vertex and the mouth vertex of the user's face, and the second line connecting the mouth vertex and the chin vertex of the user's face to obtain the maxillofacial parameters and then determine the user's face shape. Therefore, the user's face shape can be accurately determined through this simple calculation method, and the maxillofacial part of the user can be accurately adjusted according to the user's face shape.

[0091] Optionally, in the embodiment of the present application, after obtaining the maxillofacial parameters of the user in the above step C, the image processing method provided in the embodiment of the present application may include the following steps D1 and D2:

[0092] Step D1: When the angle value of the included angle belongs to a first preset angle value range, the image processing device maintains the maxillofacial parameters of the current user.

[0093] Step D2: When the angle value of the included angle exceeds the first preset angle value range, the image processing device adjusts the angle value to be within the first preset angle value range.

[0094] Exemplarily, the first preset angle value range may be preset by the electronic device or may be user-defined, which is not limited in the embodiments of the present application.

[0095] It can be understood that, in the general sense, the straight face shape is the most ideal face shape, and the angle value corresponding to the straight face shape is 180°. However, in actual applications, due to the differences in each person's facial features, when the maxillofacial parameters are not 180°, if the angle in the maxillofacial parameters is directly adjusted to 180°, the adjusted user's face shape is not the best visual effect. Therefore, a first preset angle value range can be set for the angle in the maxillofacial parameters. Within the first preset angle value range, the ideal face shape is assumed by default, and there is no need to adjust the user's maxillofacial parameters.

[0096] Example 4: Assuming that the first preset angle value range is 175° to 185°, then Figure 3 As shown, in the side face photo taken by the user, the nose vertex and the mouth vertex are connected to generate a first line 31, and the mouth vertex and the chin vertex are connected to generate a second line 32, and the degree of the angle 33 between the first line 31 and the second line 32 is 200°, then it can be determined that the user's face shape is a convex face shape, and the electronic device can adjust the face shape, adjusting the mouth inward, until the degree of the angle 33 between the first line 31 and the second line 32 is in the range of 175° to 185°.

[0097] Example 5: Combined with the above example 4, Figure 4 As shown, if the angle 43 between the first line 31 and the second line 32 is 160°, it can be determined that the user's face shape is a concave face shape, and the electronic device can adjust the face shape by adjusting the mouth outward until the angle 43 between the first line 31 and the second line 32 is within the range of 175° to 185°.

[0098] In this way, by determining the relationship between the included angle and the first preset angle value range, the user's face shape can be quickly determined, and by adjusting the mouth so that it is within the first preset angle value range, the user's face shape can be quickly and accurately corrected.

[0099] Optionally, in the embodiment of the present application, the above-mentioned maxillofacial parameters include the angle parameters of the mandibular maxillofacial area of ​​the above-mentioned user. On this basis, in the above-mentioned step 304, the image processing method provided in the embodiment of the present application may include the following step E:

[0100] Step E: The image processing device adjusts the angle parameters of the mandibular face based on preset maxillofacial parameters that match the face shape of the user.

[0101] For example, the angle parameters of the mandibular face may include angle parameters of the E-line feature points, etc. Generally, in a well-balanced profile, the lower lip is on the E-line, and the upper lip is on the line or 0.5-1 mm behind the line. The angle parameters of the E-line feature points are connected to form 180°.

[0102] In one example, when the user's face is convex, the E-line feature points will be connected to form an obvious convex point, and the image processing device can use an algorithm to shrink the user's mandibular face inward to reduce the mandibular angle; when the user's face is concave, the E-line feature points will be connected to form an obvious concave point, and the image processing device can use an algorithm to expand the mouth outward to increase the mandibular angle.

[0103] In this way, the image processing device adjusts the angle parameters of the mandibular and maxillofacial surfaces to make the user's face shape more consistent with existing aesthetic standards, thereby providing the user with a simple, direct, fast and effective image processing method.

[0104] It should be noted that the image processing method provided in the embodiment of the present application can be executed by an image processing device or a control module in the image processing device for executing the image processing method. In the embodiment of the present application, the image processing device provided in the embodiment of the present application is described by taking the image processing device executing the image processing method as an example.

[0105] Figure 5 A possible structural diagram of an image processing device provided in an embodiment of the present application is provided. Figure 5 As shown, the above-mentioned device 600 includes an acquisition module 601, a calculation module 602, a determination module 603 and an adjustment module 604; the above-mentioned acquisition module 601 is used to obtain the feature point information of the jaw feature points among the user's facial feature points; the above-mentioned calculation module 602 is used to calculate the maxillofacial parameters of the above-mentioned user according to the feature point information of the jaw feature points obtained by the above-mentioned acquisition module 601; the above-mentioned determination module 603 is used to determine the face shape of the above-mentioned user according to the above-mentioned maxillofacial parameters calculated by the above-mentioned calculation module 602; the above-mentioned adjustment module 604 is used to adjust the above-mentioned maxillofacial parameters based on preset maxillofacial parameters matching the face shape of the above-mentioned user determined by the above-mentioned determination module 603.

[0106] The image processing device provided in the embodiment of the present application can calculate the maxillofacial parameters of the user based on the feature point information of the jaw feature points in the user's facial feature points after acquiring the feature point information of the jaw feature points, and determine the user's face shape based on the jaw feature parameters, and finally determine the preset maxillofacial parameters that match the user's face shape, and then adjust the jaw parameter. In this way, by acquiring the jaw feature of the user, the parameters of the maxillofacial part of the user can be adjusted in a targeted manner, and then provide accurate and effective correction parameters for the electronic device when processing the user's facial photos at angles other than the front (for example, the user's facial photos at the side or other angles), so that the electronic device can effectively process the user's facial photos at angles other than the front.

[0107] Optionally, in an embodiment of the present application, the above-mentioned device 600 also includes: an identification module 605; the above-mentioned identification module 605 is used to identify the E-line feature points from the above-mentioned mandibular part feature points; the above-mentioned calculation module 602 is specifically used to calculate the maxillofacial parameters of the above-mentioned user based on the feature point information of the above-mentioned E-line feature points identified by the above-mentioned identification module 605.

[0108] Optionally, in an embodiment of the present application, the E-line feature points include a nose vertex, a mouth vertex and a chin vertex of the user's face, and the determination module 603 is further used to determine the mouth state of the user according to the feature point information of the facial feature points of the user acquired by the acquisition module; when the mouth state is a tooth-exposed state, the mouth vertex is the highest point of the teeth of the exposed tooth part; when the mouth state is a non-tooth-exposed state, the mouth vertex is the highest point of the lips.

[0109] Optionally, in an embodiment of the present application, the above-mentioned device 600 also includes: an acquisition module 606; the above-mentioned acquisition module 606 is used to obtain the user's maxillofacial parameters by calculating the angle between a first line connecting the nose vertex and the mouth vertex of the user's face, and a second line connecting the mouth vertex and the chin vertex of the user's face.

[0110] Optionally, in an embodiment of the present application, the above-mentioned adjustment module 604 is also used to maintain the maxillofacial parameters of the current user when the angle value of the above-mentioned angle belongs to the first preset angle value range; the above-mentioned adjustment module is also used to adjust the angle value to within the above-mentioned first preset angle value range when the angle value of the above-mentioned angle exceeds the above-mentioned first preset angle value range.

[0111] The image processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0112] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0113] The image processing device provided in the embodiment of the present application can achieve Figure 1 and Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0114] It should be noted that if Figure 5 As shown, the modules that must be included in the image processing device 600 are indicated by solid-line boxes, such as the acquisition module 601 ; the modules that may or may not be included in the image processing device 600 are indicated by dotted-line boxes, such as the recognition module 605 .

[0115] Optional, such as Figure 6 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a program or instruction stored in the memory 802 and executable on the processor 801. When the program or instruction is executed by the processor 801, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0116] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0117] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0118] The electronic device 100 includes, but is not limited to, components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110. The user input unit 107 includes a touch panel 1071 and other input devices 1072, the display unit 106 includes a display panel 1061, the input unit 104 includes an image processor 1041 and a microphone 1042, and the memory 109 can be used to store software programs (such as an operating system, an application required for at least one function) and various data.

[0119] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0120] Among them, the processor 110 is used to obtain the feature point information of the jaw feature points among the user's facial feature points; the processor 110 is also used to calculate the maxillofacial parameters of the user based on the feature point information of the jaw feature points; the processor 110 is also used to determine the face shape of the user based on the maxillofacial parameters; the processor 110 is also used to adjust the maxillofacial parameters based on preset maxillofacial parameters that match the face shape of the user.

[0121] The electronic device in the embodiment of the present application, after obtaining the feature point information of the jaw feature point in the user's facial feature points, can calculate the user's maxillofacial parameters based on the feature point information of the jaw feature point, and determine the user's face shape based on the maxillofacial parameters, and finally determine the preset maxillofacial parameters that match the user's face shape, and then adjust the maxillofacial parameters. In this way, by obtaining the jaw feature of the user, the parameters of the user's maxillofacial part can be adjusted in a targeted manner, and then provide accurate and effective correction parameters for the electronic device when processing the user's facial photos at angles other than the front (for example, the user's facial photos at the side or other angles), so that the electronic device can effectively process the user's facial photos at angles other than the front.

[0122] Optionally, the processor 110 is further used to identify E-line feature points from the above-mentioned mandibular part feature points; the processor 110 is specifically further used to calculate the maxillofacial parameters of the above-mentioned user based on the feature point information of the above-mentioned E-line feature points.

[0123] Optionally, the processor 110 is specifically used to obtain the user's maxillofacial parameters by calculating the angle between a first line connecting a nose vertex and a mouth vertex of the user's face, and a second line connecting a mouth vertex and a chin vertex of the user's face.

[0124] Optionally, the processor 110 is also used to maintain the maxillofacial parameters of the current user when the angle value of the above-mentioned angle belongs to the first preset angle value range; the processor 110 is also used to adjust the angle value to within the first preset angle value range when the angle value of the above-mentioned angle exceeds the first preset angle value range.

[0125] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include but are not limited to a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 109 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 110 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface, and the application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0126] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0127] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0129] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0130] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0132] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises, Acquire feature point information of the jaw feature points in the user's facial feature points; Calculating the maxillofacial parameters of the user according to the feature point information of the feature points of the mandibular part; Determining the face shape of the user according to the maxillofacial parameters; adjusting the maxillofacial parameters based on preset maxillofacial parameters matching the face shape of the user; The step of calculating the maxillofacial parameters of the user according to the feature point information of the mandibular feature points includes: From the jaw part feature points, identify E-line feature points, wherein the E-line feature points include a nose vertex, a mouth vertex, and a chin vertex of the user's face; Obtaining the maxillofacial parameters of the user by calculating the angle between a first line connecting the nose vertex and the mouth vertex and a second line connecting the mouth vertex and the chin vertex; After obtaining the maxillofacial parameters of the user, the method further includes: When the angle value of the included angle belongs to the first preset angle value range, maintaining the maxillofacial parameters of the current user; When the angle value of the included angle exceeds the first preset angle value range, the angle value of the included angle is adjusted to be within the first preset angle value range.

2. The method according to claim 1, characterized in that Before identifying the E-line feature points, the method further includes: Determining a mouth state of the user according to feature point information of facial feature points of the user; The method further comprises: When the mouth state is a tooth-exposed state, the mouth vertex is the highest point of the teeth where the teeth are exposed; When the mouth state is a non-teeth-exposed state, the mouth apex is the highest point of the lips.

3. An image processing device, characterized in that: The device comprises an acquisition module, a calculation module, a determination module and an adjustment module; The acquisition module is used to acquire feature point information of the jaw feature points among the facial feature points of the user; The calculation module is used to calculate the maxillofacial parameters of the user according to the feature point information of the mandibular feature points acquired by the acquisition module; The determination module is used to determine the face shape of the user according to the maxillofacial parameters calculated by the calculation module; The adjustment module is used to adjust the maxillofacial parameters based on preset maxillofacial parameters that match the face shape of the user determined by the determination module; The device also includes: an identification module and an acquisition module; The recognition module is used to recognize E-line feature points from the jaw part feature points, wherein the E-line feature points include a nose vertex, a mouth vertex, and a chin vertex of the user's face; The acquisition module is used to acquire the maxillofacial parameters of the user by calculating the angle between a first line connecting the nose vertex and the mouth vertex and a second line connecting the mouth vertex and the chin vertex; The adjustment module is further configured to, after acquiring the maxillofacial parameters of the user, maintain the maxillofacial parameters of the current user when the angle value of the included angle belongs to a first preset angle value range; The adjustment module is further configured to adjust the angle value of the included angle to within the first preset angle value range when the angle value of the included angle exceeds the first preset angle value range.

4. The device according to claim 3, characterized in that The determining module is further configured to determine the mouth state of the user according to the feature point information of the facial feature points of the user acquired by the acquiring module; When the mouth state is a tooth-exposed state, the mouth vertex is the highest point of the teeth where the teeth are exposed; When the mouth state is a non-teeth-exposed state, the mouth apex is the highest point of the lips.

5. An electronic device, characterized in that: The method comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method as claimed in claim 1 or 2.

Images