Methods, storage media and equipment for evaluating the effect of jaw lifting

Abstract

This application relates to a method, storage medium, and device for evaluating the effect of chin lifting. The method includes: using a pre-trained model to predict the face position of a face image and obtaining the coordinates of each facial feature point; cropping a standard-sized image based on the nose tip coordinates; estimating the head posture using a camera pose estimation method; calculating the coordinates of the chin feature points and obtaining the chin curve based on the coordinates of the chin feature points; determining whether the facial posture before and after chin lifting matches based on the coordinates of the left corner of the left eye, the right corner of the right eye, the nose tip, the left corner of the mouth, and the right corner of the mouth feature points; calculating and comparing the curve fitting degree, absolute area of ​​the curve, distance between curve segments, length of the chin curve, and distance between the chin and the nose tip of the chin curve before and after chin lifting, and comprehensively evaluating the chin lifting effect from these five perspectives, thereby achieving an accurate evaluation of the chin lifting effect.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, storage medium and device for evaluating the effect of chin lifting. Background Technology

[0002] The pursuit of beauty is human nature. As we age, skin sags and soft tissue accumulates in the neck area, increasing the downward pressure. Therefore, people are paying more and more attention to beauty. However, purely natural beauty treatments still show limited effectiveness, leading to the growing acceptance of minimally invasive medical procedures. Jawline lifting is currently a highly popular facial anti-aging procedure. Jawline lifting can address various skin aging problems: reducing nasolabial folds and crow's feet, lifting the face and cheekbones, improving facial contours, and creating a V-shaped face. However, the effectiveness of current jawline lifting methods varies, sometimes resulting in subtle effects, and the degree of lifting cannot be accurately judged by the naked eye. Summary of the Invention

[0003] To facilitate accurate evaluation of the jaw lifting effect, this application provides a method, storage medium, and device for evaluating the jaw lifting effect.

[0004] Firstly, this application provides a method for evaluating the effect of jaw lifting, which adopts the following technical solution:

[0005] A method for evaluating the effect of jaw lifting includes:

[0006] A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained.

[0007] Crop the image to a standard size based on the coordinates of the nose tip;

[0008] The head pose is estimated using a camera pose estimation method;

[0009] The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points.

[0010] Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches.

[0011] The effect of jaw lifting is evaluated based on the jawline before and after jaw lifting, combined with the coordinates of facial feature points.

[0012] By employing the aforementioned technical solution, images of the user before and after chin lift are captured, resulting in facial images before and after the lift. A trained model is used to predict the facial position and coordinates of facial feature points within these images. Based on the coordinates of five feature points—the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth—the facial posture before and after the lift is assessed to determine if they match, facilitating accurate subsequent judgment of the positional distances of these feature points. By analyzing the coordinates of the facial feature points and the chin curve, the difference in facial contour before and after the lift is obtained, allowing for a precise evaluation of the chin lift effect.

[0013] Optionally, before the step of predicting the face location using a pre-trained model for the face image, the method further includes: replacing the image background in the face image within a specified color gamut range, and performing image enhancement processing using a spatial domain algorithm.

[0014] By adopting the above technical solution, before predicting the face position in the face image, image processing is performed on the face image, the image background within a specified color gamut is replaced, the image background is unified, and image enhancement processing is performed to increase the contrast, thereby improving the recognition effect of face position.

[0015] Optionally, the head pose estimation using a camera pose estimation method is specifically as follows:

[0016] The OpenCV-PnP algorithm is used to estimate head pose. The rotation vector is converted into three mutually perpendicular three-dimensional angle indices: nodding, tilting, and shaking. The corresponding rotation matrix and translation vector are solved to complete the head pose estimation.

[0017] If the head posture angle does not meet the requirements when using the above technical solution, the evaluation of the chin lift effect will be invalid. By estimating the head posture, when the head posture angle is incorrect, it can be adjusted using the OpenCV-PnP algorithm to avoid affecting the subsequent evaluation of the chin lift effect.

[0018] Optionally, based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after chin lift matches, specifically:

[0019] Calculate the perimeter of the posture image composed of five feature points: the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth. Obtain the posture difference between the perimeter of the posture image after chin lift and the perimeter of the posture image before chin lift. When the posture difference is less than a preset posture threshold, it is determined that the facial posture before and after chin lift matches. Otherwise, the facial posture does not match.

[0020] By employing the aforementioned technical solution, the posture difference between the perimeter of the post-chin lift posture image and the perimeter of the pre-chin lift posture image is calculated. This posture difference is then used to determine whether the facial postures before and after the chin lift match. By judging the facial postures, the chin lift effect is evaluated only when the facial postures match, thus avoiding the problem of invalid evaluations due to posture mismatch.

[0021] Optionally, the jawline lifting effect can be evaluated based on the jawline curve and the coordinates of facial feature points, specifically:

[0022] Curve fit: Calculate the Frechet distance between the two mandibular curves before and after mandibular lifting. If the Frechet distance value is greater than the preset curve distance threshold, it is determined that there is a significant change in the mandibular curve.

[0023] Absolute area of ​​the curve: Calculate the area difference between the horizontal line of the nasal tip and the closed shape formed by the mandibular curve before and after chin lift. If the area difference is greater than 0, the chin lift effect is considered negative; otherwise, the effect is considered positive. If the area difference is less than a preset area difference threshold, the lift effect is considered significant. The preset area difference threshold is less than 0.

[0024] Distance between curve segments: The distance between the horizontal line of the tip of the nose and the horizontal line of the lower lip before and after the chin lift is calculated. If the distance is greater than a preset threshold, it is determined that there is a narrowing effect on the cheeks; otherwise, there is no effect.

[0025] Jawline curve length: Calculate the difference in jawline length below the horizontal line of the nose tip before and after jawline lifting. If the length difference is greater than 0, the effect is considered negative, and vice versa. If the length difference is less than a preset length difference threshold, the lifting effect is considered significant. The preset length difference threshold is less than 0.

[0026] Chin and nose tip distance: Calculate the Euclidean distance between the chin feature point and the nose tip before and after chin lift, and compare the difference between the Euclidean distances before and after chin lift. If the difference is less than the preset Euclidean distance threshold, the lifting effect is determined to be significant.

[0027] By adopting the above technical solutions, the curve fitting degree, absolute area of ​​the curve, distance between curve segments, length of the mandibular curve, and distance between the chin and the tip of the nose are calculated and compared before and after mandibular lifting. The mandibular lifting effect is comprehensively evaluated from these five perspectives, which can achieve a comprehensive evaluation and thus realize an accurate evaluation of the mandibular lifting effect.

[0028] Optionally, the area of ​​the closed shape formed by the horizontal line of the nasal tip and the mandibular curve before and after the mandibular lift is calculated as follows:

[0029] The problem of covering the area of ​​a closed curve is transformed into a graphical problem. OpenCV is used to fill the convex hull, the image after filling the convex hull is binarized, and NumPy is used to calculate the area of ​​the closed image.

[0030] By adopting the above technical solution, the area of ​​the closed shape formed by the horizontal line of the nasal tip and the curve of the mandible can be accurately calculated using the convex hull algorithm. This facilitates the calculation of the area difference of the corresponding closed shape before and after mandibular lifting, thereby determining whether the mandibular lifting effect is significant.

[0031] Optionally, the mandibular feature points include 17 facial feature points centered at the midpoint below the mandibular edge and adjacent to it.

[0032] By adopting the above technical solution, 17 mandibular feature points were selected with the midpoint below the mandibular edge as the center and adjacent to it. The mandibular curve formed by these 17 mandibular feature points can clearly represent the mandibular edge before and after mandibular lifting.

[0033] Optionally, the chin feature points include nine facial feature points centered at the midpoint below the jawline and adjacent to it.

[0034] By employing the aforementioned technical solution, nine chin feature points were selected, centered on the midpoint below the jawline and adjacent to it. These nine chin feature points clearly indicate the position of the chin edge. By calculating the average Euclidean distance between these nine chin feature points and the tip of the nose, and comparing the difference between the average Euclidean distances before and after chin lifting, it is possible to accurately determine whether a lifting effect has been achieved in the chin area.

[0035] Secondly, this application provides a device for evaluating the effect of jaw lifting, which adopts the following technical solution: a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to achieve the following:

[0036] A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained.

[0037] Crop the image to a standard size based on the coordinates of the nose tip;

[0038] The head pose is estimated using a camera pose estimation method;

[0039] The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points.

[0040] Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches.

[0041] The steps for evaluating the chin lift effect are based on the chin curves before and after the chin lift, combined with the coordinates of the facial feature points.

[0042] By adopting the above technical solution, the computer device can implement the method for evaluating the jaw lifting effect provided in this application.

[0043] Thirdly, this application provides a storage medium for evaluating the effect of jaw lifting, which adopts the following technical solution:

[0044] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor at runtime:

[0045] A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained.

[0046] Crop the image to a standard size based on the coordinates of the nose tip;

[0047] The head pose is estimated using a camera pose estimation method;

[0048] The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points.

[0049] Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches.

[0050] The steps for evaluating the chin lift effect are based on the chin curves before and after the chin lift, combined with the coordinates of the facial feature points.

[0051] By adopting the above technical solution, a computer program is stored on the computer-readable storage medium, and the computer program, when run by a processor, can implement the method for evaluating the jaw lifting effect provided in this application.

[0052] In summary, this application includes the following beneficial technical effects:

[0053] This application predicts the position of the face in a face image, obtains the coordinates of the face feature points, determines the facial posture before and after chin lift, and then comprehensively evaluates the chin lift effect from multiple angles based on the coordinates of the face feature points and the chin curve, thereby achieving a precise evaluation of the chin lift effect. Attached Figure Description

[0054] Figure 1 This is a flowchart of a method for evaluating the jaw lifting effect according to one embodiment of this application.

[0055] Figure 2 This is a schematic diagram illustrating a multi-angle evaluation of the jaw lifting effect according to one embodiment of this application. Detailed Implementation

[0056] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figure 1-2 The present application will be further described in detail with reference to the embodiments.

[0057] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0058] like Figure 1 As shown in the embodiment of this application, a method for evaluating the effect of jaw lifting is disclosed.

[0059] 1. Use a pre-trained model to predict the face location for the face image and obtain the coordinates of each face feature point.

[0060] (1) Take facial images of the user before and after chin lift, and classify the captured facial images.

[0061] (2) Before performing face location prediction, perform image enhancement processing on the face image.

[0062] Image enhancement techniques can be broadly categorized into frequency domain-based algorithms and spatial domain-based algorithms, depending on the spatial context of the enhancement process. Frequency domain-based algorithms modify the transform coefficients of an image within a specific transform domain, representing an indirect enhancement approach. They treat the image as a two-dimensional signal and perform signal enhancement based on a two-dimensional Fourier transform. Low-pass filtering removes noise, while high-pass filtering enhances high-frequency signals such as edges, making blurry images clearer. Spatial domain-based algorithms include point operation algorithms and neighborhood enhancement algorithms. Point operation algorithms include grayscale transformation and histogram correction, aiming to either achieve uniform image quality or expand the dynamic range and contrast. Neighborhood enhancement algorithms include image smoothing and image sharpening. Image smoothing is generally used to eliminate image noise but can easily blur edges; common algorithms include mean filtering and median filtering. Image sharpening aims to highlight the edges and contours of objects for easier target recognition; common algorithms include gradient methods, operators, high-pass filtering, mask matching, and statistical interpolation. While a single method can be used for enhancement, in most practical cases, several methods need to be used in combination to achieve the desired enhancement effect.

[0063] In this embodiment, a spatial domain-based algorithm is specifically used to enhance the facial image. First, the background of the image within a specified color gamut is replaced, replacing the background of the facial images before and after chin lift with a uniform background. Then, a histogram correction method is used to transform the facial image after background replacement, correcting the grayscale histogram of the facial image to a uniform histogram. Image smoothing and image sharpening methods are then used to smooth and sharpen the histogram-corrected facial image, filtering out image noise and enhancing the facial edge contours, making the facial edges and various parts clearer, thus improving the facial recognition effect.

[0064] (3) Use the pre-trained model to predict the face position and obtain the coordinates of each face feature point.

[0065] To achieve face recognition, face alignment is a fundamental algorithm. Currently, the main algorithms include ERT, SDM, and LBF. Among them, the dlib open-source library implements the ERT cascade regression algorithm, which has good results. Here, we use dlib and implement it through Python.

[0066] In this embodiment, a pre-trained keypoint extraction model provided by the dilb library in Python is used to implement face detection. The ERT cascade regression algorithm is used to extract facial feature points. This model can identify 68 facial feature points. Among these 68 facial feature points, 17 feature points are used to predict the outer contour of the face, namely 17 jaw feature points, and the remaining 51 feature points are facial feature points.

[0067] Sometimes face images are taken from different angles, so a face may not be detected. Therefore, if a face is not detected, rotate the image by 60 degrees in sequence and try again until a face is detected. If a face is still not detected after rotating 360 degrees, then it is considered that there is no face in the image.

[0068] 2. Based on the coordinates of the nose tip, crop the image to a standard size.

[0069] Based on the coordinates of the nasal tip feature point in the facial feature points before and after chin lift, the facial regions containing 68 facial feature points in the facial images before and after chin lift are cropped to standard sizes. This cropping of the facial regions facilitates subsequent comparison of the chin curve before and after chin lift, making the comparison results more accurate. The standard size can be pre-set or determined in real-time based on a set standard image size and the original image size.

[0070] 3. The head pose is estimated using the camera pose estimation method.

[0071] Camera pose estimation involves determining the camera's coordinates and rotation angles within a coordinate system by using several feature points with known coordinates and their images within the camera. This embodiment employs the OpenCV-PnP algorithm to estimate head pose, converting the rotation vector into three mutually perpendicular three-dimensional angle indices: nodding, tilting, and shaking. The corresponding rotation matrix and translation vector are then calculated to complete the head pose estimation.

[0072] The PnP algorithm, short for Perspective-n-Point, works by estimating the camera pose when n three-dimensional spatial points and their two-dimensional projection positions are known. Specifically, using the OpenCV-PnP algorithm, the rotation matrix R and translation vector t corresponding to the pixel coordinates and physical coordinates are solved, thus completing the camera calibration. The rotation matrix R is shown in formula (1):

[0073]

[0074] The rotation angles of each axis can be calculated using the rotation matrix R.

[0075] Z-axis rotation angle θ z for:

[0076] θ z =atan2(r 21 ,r 11 (2).

[0077] Y-axis rotation angle θ y for:

[0078]

[0079] X-axis rotation angle θ x for:

[0080] θ x =atan2(r 32 ,r 33 (4).

[0081] Based on the OpenCV-PnP algorithm and the rotation matrix R, the translation vector t from the object coordinate system to the camera coordinate system is determined.

[0082] 4. Calculate the coordinates of the mandibular feature points using an interpolation algorithm, and obtain the mandibular curve using the shortest path algorithm based on the coordinates of the mandibular feature points.

[0083] Interpolation is the process of finding intermediate values ​​from a set of discrete data. For example, when looking up data in a logarithmic table, if the data you are looking for cannot be found in the table, you first find its adjacent numbers, then find its correction value next to it in the table, and correct these adjacent numbers according to a certain relationship to find the number you are looking for. This correction relationship is, in fact, a kind of interpolation.

[0084] The purpose of the interpolation algorithm is to find a sufficiently smooth and relatively simple function g(c) (called the interpolation function) as an approximate expression of k(c) based on the value of the function k(c) at the node (the functional form of k(c) is unknown), and then calculate the value of c of g(c) in the interpolation interval as the approximate value of the original function k(c) at this point.

[0085] Of the 68 facial feature points mentioned, not all feature point coordinates are needed when evaluating the jawline lifting effect. In this embodiment, 17 facial feature points are specifically used, namely, the 17 adjacent facial feature points centered at the midpoint below the jawline. A bilinear interpolation algorithm is used to obtain the precise coordinates of these 17 jawline feature points, resulting in a smoother jawline curve.

[0086] Based on the coordinates of the 17 mandibular feature points, the mandibular curve is obtained using a shortest path algorithm. This embodiment uses Dijkstra's algorithm to calculate the mandibular curve. Dijkstra's algorithm is a typical shortest path algorithm used to calculate the shortest path from one node to other nodes. Its main characteristic is that it expands outward layer by layer from the starting point (breadth-first search) until it reaches the destination. The specific algorithm is as follows:

[0087] When calculating the shortest path using Dijkstra's algorithm, specify the starting point s (i.e., start the calculation from vertex s);

[0088] Introduce two sets S and U. Set S records the vertices for which the shortest path has been found (and the corresponding shortest path length), while set U records the vertices for which the shortest path has not yet been found (and the distance from the vertex to the starting point s).

[0089] Initially, set S contains only the starting point s; set U contains all vertices except s, and the path of a vertex in set U is the path from the starting point s to that vertex. Find the vertex with the shortest path in set U and add it to set S; update the vertices in set U and their corresponding paths; then find the vertex with the shortest path in set U again and add it to set S; then update the vertices in set U and their corresponding paths; ..., repeat this operation until all vertices have been traversed, and finally obtain the mandibular curve.

[0090] 5. Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth among the facial feature points, determine whether the facial posture before and after the chin lift matches.

[0091] Images of the face taken before and after a chin lift may contain inconsistencies in facial expressions, rendering the evaluation of the lift's effectiveness invalid. Therefore, it is necessary to determine whether the facial posture matches before and after the images are taken before evaluating the chin lift effect.

[0092] This embodiment uses five feature points out of the 68 facial feature points—the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth—to determine whether the facial posture matches. Based on the coordinates of these five feature points, the perimeter of the posture image formed by these five feature points is calculated, resulting in the posture difference between the perimeter of the posture image after chin lift and the perimeter of the posture image before chin lift. When the posture difference is less than a preset posture threshold, the facial posture before and after chin lift is considered to match; if the posture difference is greater than the preset posture threshold, the facial posture is considered to be mismatched. By judging whether the facial posture matches, the problem of invalid evaluation due to posture mismatch can be avoided.

[0093] 6. Evaluate the chin lift effect based on the chin curves before and after the chin lift, combined with the coordinates of facial feature points.

[0094] like Figure 2 As shown, the evaluation will be conducted from the following five perspectives.

[0095] (1) Curve fitting degree: Calculate the Frechet distance between the two mandibular curves before and after mandibular lifting. If the Frechet distance value is greater than the preset curve distance threshold, it is determined that there is a significant change in the mandibular curve.

[0096] The significance of the change in the jawline curve is determined by comparing the distance between the two jawline curves before and after the jawline lift. The Frechet distance algorithm is used to calculate the distance between two curves and determine their similarity. It calculates the distance between points on the two curves, pairs these points according to their distance and the curve's trend, and finally selects the discrete Frechet distance based on these pairing distances. A smaller Frechet distance indicates a higher similarity.

[0097] The Frechet Distance algorithm is as follows:

[0098] Let the pair (D, d) be a metric space, where d is a metric function on D.

[0099] If the mapping γ:[0,1]→D on the unit interval [0,1] is a continuous mapping, then γ is called a continuous curve on D.

[0100] Let the mandibular curve A before mandibular lift and the mandibular curve B after mandibular lift be two curves on D, i.e., A:[0,1]→D, B[0,1]→D. Let α and β be two reparameterized functions within a unit interval, i.e., α:[0,1]→[0,1], β:[0,1]→[0,1]. Then the Frechet distance H(A,B) between mandibular curves A and B is defined as:

[0101]

[0102] (2) Absolute area of ​​the curve: Calculate the area difference between the closed shape formed by the horizontal line of the nasal tip and the mandibular curve before and after chin lift. The area difference is obtained by subtracting the area of ​​the closed shape before chin lift from the area of ​​the closed shape after chin lift. When the area difference is greater than 0, it indicates that the chin lift effect is negative; when the area difference is equal to 0, it indicates that there is no chin lift effect; when the area difference is less than 0, it indicates that there is a chin lift effect; when the area difference is less than a preset area difference threshold, it indicates that the chin lift effect is significant, wherein the preset area difference threshold is less than 0.

[0103] The area of ​​the closed figure formed by the horizontal line of the nasal tip and the curve of the mandible is calculated as follows:

[0104] To calculate the area of ​​a closed shape formed by the horizontal line of the nose tip and the curve of the jaw, the area of ​​the closed curve is transformed into a graphical problem. The convex hull function provided by OpenCV is used to fill the convex hull. The filled image is then binarized, and the number of convex hulls is counted using NumPy.

[0105] A convex hull is a concept in computer geometry. Simply put, given a set of points in a two-dimensional plane, the convex hull is the smallest convex polygon that can be formed by connecting the outermost points. Convex hull detection is used in object recognition, gesture recognition, and boundary detection. In a closed shape formed by the horizontal line of the nose tip and the curve of the jaw, given a set P, the intersection of all convex sets containing P is called the convex hull of P. The convex hull of P can be constructed using a linear combination of all points (p0, p1, ..., pn) within P. Given a set P containing all points (p0, p1, ..., pn), we need to find the convex hull of P in an image. Using OpenCV's `convexHull` function, we can draw this convex hull and obtain a visual representation of the result.

[0106] Image binarization involves setting a pre-defined threshold. When the grayscale value of an image pixel is greater than the threshold, the grayscale value is set to 255; when the grayscale value is less than the threshold, the grayscale value is set to 0. This results in a black and white image that is either 0 or 255, and such an image is called a binary image. The outline of the convex hull is more clearly displayed in a binary image.

[0107] This section uses NumPy to calculate the area of ​​the convex hull in a binary image. The path describing each contour is obtained from the `contour` attribute of the convex hull contour set returned by the `collections` function. Then, the `vertices` attribute of the path contains the ordered vertices of the contour. Using these vertices, the contour integral 0.5*(x*dy-y*dx) is approximated. By applying Green's theorem, the area of ​​the closed shape formed by the horizontal line of the nose tip and the chin curve is obtained.

[0108] (3) Distance between curve segments: The mandibular curve segments between the horizontal line of the tip of the nose and the horizontal line of the lower lip before and after the mandibular lift are respectively extracted, and the distance between the mandibular curve segments before and after the mandibular lift is calculated. If the distance is greater than a preset threshold, it is determined that there is a narrowing effect on the cheek area; otherwise, there is no effect.

[0109] This embodiment evaluates the lifting effect on the entire jawline and cheek area. The jawline curve segment between the horizontal coordinate line of the nose tip and the horizontal coordinate line of the center point of the lower lip is extracted. Using the Frechet Distance algorithm, the distance between the jawline curve segments at the same position before and after jawline lifting is calculated. When this distance is greater than a preset threshold, it is determined that the cheek area has a narrowing effect.

[0110] (4) Mandibular curve length: Calculate the difference in mandibular curve length below the horizontal line of the nose tip before and after mandibular lifting. If the length difference is greater than 0, the effect is considered negative, otherwise it is considered positive. If the length difference is less than the preset length difference threshold, the lifting effect is determined to be significant.

[0111] This embodiment evaluates the lifting effect on the entire chin area. The chin curve below the horizontal line where the nose tip is located is taken as the chin curve. The lengths of the chin curve after lifting and before lifting are calculated. The difference between the lengths of the chin curve after lifting and before lifting is calculated. When the length difference is greater than 0, the lifting effect on the chin is negative; when the length difference is equal to 0, there is no lifting effect; when the length difference is less than 0, there is a lifting effect; and when the length difference is less than a preset threshold, the lifting effect on the chin is significant, where the preset length difference threshold is less than 0.

[0112] The calculation method for the length of the chin curve after chin lift and the length of the chin curve before chin lift is as follows: Let f(x) be the chin curve after chin lift, and v and w be the two endpoints of the curve. The chin curve is a smooth and continuous curve. According to the idea of ​​differentiation, the length of a section of the curve can be divided into the sum of countless short curves.

[0113] Suppose we divide the interval [v, w] into n subintervals using n-1 numbers, then the arc length Δl of each subinterval is... i It can be expressed by the following formula:

[0114]

[0115] Where, x i Let i be any point within the i-th subinterval, where i = 1, 2, ..., n;

[0116] The total arc length L of the curve is approximately equal to the sum of the arc lengths of each subinterval:

[0117]

[0118] (5) Distance between chin and nose tip: Calculate the Euclidean distance between the chin feature point and the nose tip before and after chin lifting, compare the difference between the Euclidean distances before and after chin lifting, and if the difference is less than the preset Euclidean distance threshold, the lifting effect is determined to be significant.

[0119] Euclidean distance, also known as Euclidean distance, is used to calculate the absolute distance between two points in multidimensional space. It can also be understood as the true distance between two points in multidimensional space, or the natural length of a vector (i.e., the distance from that point to the origin). In multidimensional space, Euclidean distance is simply the actual distance between two points.

[0120] The Euclidean distance formula in two-dimensional space is:

[0121]

[0122] Where ρ is the Euclidean distance between points (o1, q1) and (o2, q2).

[0123] This embodiment evaluates the lifting effect on the chin. The chin feature points include nine facial feature points centered at the midpoint below the jawline and adjacent to each other. When calculating the distance between the chin and the tip of the nose, the Euclidean distances from each of these nine facial feature points to the tip of the nose need to be calculated separately. Finally, the average of the nine Euclidean distances is obtained, and this average distance is used as a comprehensive index to evaluate the lifting effect on the chin.

[0124] This application also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following:

[0125] A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained.

[0126] Crop the image to a standard size based on the coordinates of the nose tip;

[0127] The head pose is estimated using a camera pose estimation method;

[0128] The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points.

[0129] Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches.

[0130] The steps for evaluating the chin lift effect are based on the chin curves before and after the chin lift, combined with the coordinates of the facial feature points.

[0131] This application also discloses a computer-readable storage medium storing a computer program, which is executed by a processor:

[0132] A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained.

[0133] Crop the image to a standard size based on the coordinates of the nose tip;

[0134] The head pose is estimated using a camera pose estimation method;

[0135] The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points.

[0136] Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches.

[0137] The steps for evaluating the chin lift effect are based on the chin curves before and after the chin lift, combined with the coordinates of the facial feature points.

[0138] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for evaluating the effect of jaw lifting, characterized in that, include: A pre-trained model is used to predict the location of a face in a face image, and the coordinates of each facial feature point are obtained. Crop the image to a standard size based on the coordinates of the nose tip; The head pose is estimated using a camera pose estimation method; The coordinates of the mandibular feature points are calculated using an interpolation algorithm, and the mandibular curve is obtained using a shortest path algorithm based on the coordinates of the mandibular feature points. Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, determine whether the facial posture before and after the chin lift matches. The jawline curve before and after jawline lifting, combined with the coordinates of the facial feature points, is used to evaluate the jawline lifting effect. The jawline lifting effect is evaluated based on the jawline curves before and after the jawline lifting, combined with the coordinates of the facial feature points. Specifically: Curve fit: Calculate the Frechet distance between the two mandibular curves before and after mandibular lifting. If the Frechet distance value is greater than the preset curve distance threshold, it is determined that there is a significant change in the mandibular curve. Absolute area of ​​the curve: Calculate the area difference between the closed shape formed by the horizontal line of the nasal tip and the mandibular curve before and after chin lift. The area difference is obtained by subtracting the area of ​​the closed shape before chin lift from the area of ​​the closed shape after chin lift. If the area difference is greater than 0, the chin lift effect is determined to be negative, and vice versa. If the area difference is less than a preset area difference threshold, the lift effect is determined to be significant. Distance between curve segments: The distance between the horizontal line of the tip of the nose and the horizontal line of the lower lip before and after the chin lift is calculated. If the distance is greater than a preset threshold, it is determined that there is a narrowing effect on the cheeks; otherwise, there is no effect. Mandibular curve length: Calculate the difference in mandibular curve length below the horizontal line of the nose tip before and after mandibular lifting. The length difference is obtained by subtracting the length of the mandibular curve below the horizontal line of the nose tip before mandibular lifting from the length of the mandibular curve below the horizontal line of the nose tip after mandibular lifting. If the length difference is greater than 0, the effect is considered negative, and vice versa. If the length difference is less than a preset length difference threshold, the lifting effect is considered significant. Chin and nose tip distance: Calculate the Euclidean distance between the chin feature point and the nose tip before and after chin lift. Subtract the Euclidean distance between the chin feature point and the nose tip before chin lift from the Euclidean distance after chin lift to obtain the difference in distance. Compare the difference in Euclidean distance before and after chin lift. If the difference is less than a preset Euclidean distance threshold, the lifting effect is considered significant.

2. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, The step of predicting face location using a pre-trained model for a face image also includes: Replace the background of the face image within a specified color gamut range, and perform image enhancement processing using a spatial domain algorithm.

3. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, The head pose estimation method using camera pose estimation is specifically as follows: The OpenCV-PnP algorithm is used to estimate head pose, converting the rotation vector into three mutually perpendicular three-dimensional angle indices: nodding, tilting, and shaking. The corresponding rotation matrix and translation vector are then obtained to complete the head pose estimation.

4. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, Based on the coordinates of the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth, it is determined whether the facial posture before and after the chin lift matches. Specifically: Calculate the perimeter of the posture image composed of five feature points: the left corner of the left eye, the right corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth. Obtain the posture difference between the perimeter of the posture image after chin lift and the perimeter of the posture image before chin lift. When the posture difference is less than a preset posture threshold, it is determined that the facial posture before and after chin lift matches. Otherwise, the facial posture does not match.

5. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, The area of ​​the closed shape formed by the horizontal line of the nasal tip and the mandibular curve before and after the chin lift is calculated as follows: The problem of covering the area of ​​a closed curve is transformed into a graphical problem. OpenCV is used to fill the convex hull, the image after filling the convex hull is binarized, and NumPy is used to calculate the area of ​​the closed curve.

6. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, The mandibular feature points include 17 facial feature points centered at the midpoint below the mandibular edge and adjacent to each other. The mandibular curve formed by the 17 mandibular feature points clearly represents the mandibular edge before and after mandibular lifting.

7. The method for evaluating the jaw lifting effect according to claim 1, characterized in that, The chin feature points include nine facial feature points centered at the midpoint below the jawline and adjacent to each other. These nine chin feature points clearly indicate the position of the chin edge.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the method described in any one of claims 1-7.

Images