Face image processing method, apparatus, device, storage medium and program product

Abstract

The application relates to a face image processing method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring a plurality of three-dimensional face image data of a target object collected at different times in a cosmetic process; determining a local face space body corresponding to the same face area from each three-dimensional face image data; acquiring the centroid of the local face space body corresponding to each three-dimensional face image data; and determining the face change of the same face area in the cosmetic process according to the difference between the centroids. The method can improve the accuracy of determining the face change in the cosmetic process.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a facial image processing method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of cosmetic medicine and anti-aging technology, facial lifting and firming has become one of the core concerns for users in skincare and medical aesthetics. Therefore, how to evaluate the effect of facial lifting and firming has become an important issue, directly affecting users' trust and experience. In order to meet users' high demands for skincare and medical aesthetics, there is a need to provide a scientific and accurate method for evaluating the effect of facial lifting and firming, so as to track facial changes in real time during skincare and medical aesthetic processes and provide the beauty industry with a standardized quantitative tool.

[0003] Traditional techniques use visual comparison to assess the effects of facial lifting and tightening. For example, they rely on before-and-after photos and the doctor or user makes a subjective judgment on the changes. This method lacks objective quantitative evidence and is easily influenced by experience and subjective emotions, making it impossible to accurately determine the changes in the face during the cosmetic procedure. Summary of the Invention

[0004] Based on this, this application provides a facial image processing method, apparatus, computer device, computer-readable storage medium, and computer program product, which can improve the accuracy of determining facial changes during the beauty process.

[0005] On the one hand, this application provides a facial image processing method, including:

[0006] Acquire multiple 3D facial image data of the target object at different times during the beauty process;

[0007] Determine the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data;

[0008] Obtain the centroid of the local facial spatial volume corresponding to each of the three-dimensional facial image data;

[0009] Based on the differences in the centroids, the facial changes in the same facial area during the beauty process are determined.

[0010] In one embodiment, determining the facial changes of the same facial region during the cosmetic procedure based on the differences in the centroids includes:

[0011] Based on the spatial positional changes between the centroids in the reference coordinate system of the three-dimensional facial image data, and the changes in the angles between the centroids and the coordinate axes of the reference coordinate system, the facial changes of the same facial region during the cosmetic procedure are determined.

[0012] In one embodiment, determining the local facial space volume corresponding to the same facial region from each of the three-dimensional facial image data includes:

[0013] Identify key facial points in the three-dimensional facial image data;

[0014] The three-dimensional facial image data is divided based on the facial key points to obtain multiple facial regions;

[0015] From the plurality of facial regions in each of the three-dimensional facial image data, a local facial spatial volume corresponding to the same facial region is determined.

[0016] In one embodiment, determining a local facial space volume corresponding to the same facial region from the plurality of facial regions of each of the three-dimensional facial image data includes:

[0017] From the multiple facial regions of the three-dimensional facial image data, extract facial mesh data corresponding to the same facial region;

[0018] Based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data, a closed geometry is constructed as the local facial space volume.

[0019] In one embodiment, before constructing the closed geometry using a reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data, the method further includes:

[0020] Reference points are determined from the three-dimensional facial image data based on the spatial positional relationship between the key facial points in the three-dimensional facial image data.

[0021] Using the reference point as the origin of the coordinate system, a three-dimensional coordinate system for the three-dimensional facial image data is constructed, which serves as the reference coordinate system.

[0022] In one embodiment, acquiring multiple three-dimensional facial image data of the target object at different times during the beauty procedure includes:

[0023] Obtain three-dimensional facial models of the target object at different times during the beauty treatment process;

[0024] The three-dimensional facial models are aligned in three dimensions, and then rendered uniformly to a preset viewpoint to obtain the three-dimensional facial image data corresponding to each three-dimensional facial model.

[0025] On the one hand, this application also provides a facial image processing apparatus, including:

[0026] The data acquisition module is used to acquire multiple three-dimensional facial image data of the target object at different times during the beauty process;

[0027] The region determination module is used to determine local facial spatial volumes corresponding to the same facial region from each of the three-dimensional facial image data;

[0028] The centroid determination module is used to obtain the centroid of the local facial space volume corresponding to each of the three-dimensional facial image data.

[0029] The effect determination module is used to determine the facial changes of the same facial area during the beauty process based on the differences of the centroids.

[0030] On the one hand, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0031] Acquire multiple 3D facial image data of the target object at different times during the beauty process;

[0032] Determine the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data;

[0033] Obtain the centroid of the local facial spatial volume corresponding to each of the three-dimensional facial image data;

[0034] Based on the differences in the centroids, the facial changes in the same facial area during the beauty process are determined.

[0035] On the one hand, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0036] Acquire multiple 3D facial image data of the target object at different times during the beauty process;

[0037] Determine the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data;

[0038] Obtain the centroid of the local facial spatial volume corresponding to each of the three-dimensional facial image data;

[0039] Based on the differences in the centroids, the facial changes in the same facial area during the beauty process are determined.

[0040] On the one hand, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0041] Acquire multiple 3D facial image data of the target object at different times during the beauty process;

[0042] Determine the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data;

[0043] Obtain the centroid of the local facial spatial volume corresponding to each of the three-dimensional facial image data;

[0044] Based on the differences in the centroids, the facial changes in the same facial area during the beauty process are determined.

[0045] The aforementioned facial image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire multiple three-dimensional facial image data of a target object collected at different times during a beauty treatment; determine local facial spatial volumes corresponding to the same facial region from each three-dimensional facial image data; obtain the centroid of the local facial spatial volume corresponding to each three-dimensional facial image data; and determine the facial changes of the same facial region during the beauty treatment based on the differences in the centroids. Thus, by acquiring multiple three-dimensional facial image data of a target object collected at different times during a beauty treatment, the three-dimensional structural changes of facial tissues can be accurately reflected through the depth dimension information of the target object's face; and by modeling and analyzing the centroids of the local facial spatial volumes corresponding to the same facial region in each three-dimensional facial image data, and determining the facial changes based on the differences in the centroids, the actual movement of facial tissues in the three-dimensional direction can be accurately captured, thereby enabling real-time and accurate determination of facial changes during the beauty treatment process. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is an application environment diagram of a facial image processing method in one embodiment;

[0048] Figure 2This is a flowchart illustrating a facial image processing method in one embodiment;

[0049] Figure 3 This is a flowchart illustrating another facial image processing method in one embodiment;

[0050] Figure 4 This is a flowchart illustrating a facial image processing method in another embodiment;

[0051] Figure 5 This is a schematic diagram of three-dimensional facial image data in one embodiment;

[0052] Figure 6 This is a structural block diagram of a facial image processing device in one embodiment;

[0053] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] Among related technologies, methods for evaluating the effects of facial lifting and tightening may include the following: visual comparison method, which relies on pre- and post-operative photos and is judged subjectively by doctors or users, but lacks objective quantitative basis and is easily affected by experience and subjective emotions; two-dimensional key point analysis method, which extracts key points in facial images and calculates the changes in horizontal and vertical distances, but due to the lack of depth dimension information, it cannot accurately reflect the three-dimensional structural changes of facial tissues; skin elasticity measurement method, which measures the skin's resilience after pressure to assess tightness, but this type of method focuses on local skin surface deformation and is difficult to systematically reflect the overall direction of soft tissue and the lifting direction.

[0056] The aforementioned methods all lack a unified spatial reference coordinate system, making accurate comparisons of individuals over time difficult. They also fail to track changes in the spatial position of local tissues, making it hard to distinguish the degree of deformation in different areas. They generally suffer from strong subjectivity, lack of three-dimensional features, lack of spatial reference, and difficulty in clearly defining the lifting direction. Therefore, there is an urgent need for a lifting and firming effect evaluation method based on three-dimensional data, with a unified reference coordinate system, and supporting automatic analysis, to achieve a more scientific, accurate, and comparable assessment of facial lifting and firming effects.

[0057] The facial image processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on another network server. Terminal 102 acquires multiple three-dimensional facial image data of the target object collected at different times during the beauty process; terminal 102 determines the local facial spatial volume corresponding to the same facial region from each three-dimensional facial image data; terminal 102 acquires the centroid of the local facial spatial volume corresponding to each three-dimensional facial image data; terminal 102 determines the facial changes of the same facial region during the beauty process based on the differences in the centroids.

[0058] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0059] In one exemplary embodiment, such as Figure 2 As shown, a facial image processing method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps S202 to S208. Wherein:

[0060] Step S202: Acquire multiple three-dimensional facial image data of the target object at different times during the beauty process.

[0061] The target audience may include users who need facial change assessment, such as those who use skincare products, receive cosmetic treatments, or use beauty devices for skincare and cosmetic procedures.

[0062] The multiple 3D facial image data can include multiple 3D facial image data of the same target object collected at different times during the beauty process. This can include 3D facial image data collected before and after skincare and cosmetic procedures, or multiple 3D facial image data collected at different times throughout the entire skincare and cosmetic procedure. For example, to evaluate the effectiveness of skincare products, 3D facial image data of the target object can be collected at different times: before use, after two weeks of continuous use, and after eight weeks of continuous use.

[0063] The three-dimensional facial image data can be a dataset containing complete spatial geometric information of the face, recording not only the planar shape of the facial contour (such as the position of the corners of the eyes and the wings of the nose), but also depth dimension information (such as the degree of protrusion of the cheeks and the three-dimensional curvature of the jawline). Optionally, the three-dimensional facial image data can be represented as the three-dimensional spatial coordinates of each point on the face and the triangular mesh distribution of the face.

[0064] In practical applications, multiple three-dimensional facial image data of the target subject can be acquired at different times during the beauty process using skin detection devices. For example, skin detection devices may include facial 3D scanning devices. During the acquisition process, the target subject can maintain a natural and relaxed expression, avoiding temporary facial deformations caused by facial muscle activities such as smiling or frowning.

[0065] Step S204: Determine the local facial spatial volume corresponding to the same facial region from each three-dimensional facial image data.

[0066] Among them, the same facial region may include fixed physiological sub-regions of the target object's own face, that is, facial tissues corresponding to the same position and range in multiple three-dimensional facial image data, such as the left cheek, right cheek, left mandibular border, right mandibular border, etc.

[0067] Among them, the local facial space volume can be a three-dimensional closed solid structure corresponding to the same facial region extracted from complete three-dimensional facial image data.

[0068] In practical applications, 3D facial models of the same target object acquired at different times can be 3D aligned to ensure pose consistency and angle comparability in subsequent calculations, and uniformly rendered to a predetermined viewpoint to obtain 3D facial image data corresponding to different times. Facial key points, such as the corners of the eyes, the tip of the nose, the alar of the nose, and the midpoint of the root of the nose, are extracted from the 3D facial image data. Then, based on the facial key points of the tip of the nose and the corner of the eye, a fixed reference point R is selected to construct a stable local reference coordinate system. The X, Y, and Z axes are defined as follows: X to the right, Y upward, and Z forward. Point R and its coordinate system are used as the reference datum for centroid calculation. Then, based on the standard key points of the corner of the eye, the alar of the nose, and the midpoint of the root of the nose, the left and right cheek regions are delineated, and their corresponding triangular meshes are extracted to construct a closed volume between the reference plane, thus obtaining the local facial space volume.

[0069] Step S206: Obtain the centroid of the local facial space volume corresponding to each three-dimensional facial image data.

[0070] The center of mass can refer to the geometric center of mass or the volumetric center of mass. The center of mass can also be the center of mass of a local spatial volume, a point on a material system where the mass is considered to be concentrated.

[0071] For example, the local facial space can be discretized into n volume units, assuming the volume of the i-th volume unit is V. i Its local centroid coordinates are (x i y i , z i ); where the local centroid coordinates of a volume element can be the geometric center of the volume element, i.e., the average of the coordinates of all vertices of the volume element. Therefore, the centroid coordinates of the local facial space volume It can be represented as:

[0072] .

[0073] In practical applications, the geometric centroid coordinates of the three-dimensional volume of a local facial space can be obtained. Then, the centroid is transformed into a reference coordinate system constructed based on the reference point R, and the three-dimensional spatial coordinates P(x, y, z) of the centroid in the reference coordinate system are recorded. The angles between the centroid and the X, Y, and Z axes of the reference coordinate system are calculated. .

[0074] Step S208: Based on the differences in centroids, determine the facial changes in the same facial area during the beauty process.

[0075] The changes in facial features may include the lifting and tightening effects of skincare, cosmetic procedures, and other beauty treatments. They may also include the direction of facial tissue movement, such as upward, inward, or improved symmetry.

[0076] In practice, changes in the position and angle of the centroid can be used as quantitative indicators to determine whether local facial tissues show an upward lifting or inward converging trend. This can be used to evaluate the immediate or long-term effects of skincare or cosmetic treatments and obtain the facial changes reflected through the same facial area.

[0077] For example, the differences between the centroids can include the differences between the three-dimensional spatial coordinates of each centroid in the reference coordinate system, thereby accurately representing the overall spatial position of the entire local facial space through the three-dimensional spatial coordinates of the centroids. For instance, taking the right cheek as an example, if the Y-axis coordinate of the centroid of the local facial space corresponding to the right cheek increases, it indicates that the right cheek tissue corresponding to this local facial space has moved upward as a whole; if the X-axis coordinate of the centroid of the local facial space corresponding to the right cheek decreases, it indicates that the right cheek tissue corresponding to this local facial space has converged towards the midline of the face as a whole.

[0078] The aforementioned facial image processing method acquires multiple three-dimensional facial image data of the target object at different times during the beauty process; determines the local facial spatial volume corresponding to the same facial region from each three-dimensional facial image data; obtains the centroid of the local facial spatial volume corresponding to each three-dimensional facial image data; and determines the facial changes of the same facial region during the beauty process based on the differences in the centroids. Thus, by acquiring multiple three-dimensional facial image data of the target object at different times during the beauty process, the three-dimensional structural changes of facial tissues can be accurately reflected through the depth dimension information of the target object's face; and by modeling and analyzing the centroids of the local facial spatial volumes corresponding to the same facial region in each three-dimensional facial image data, and determining the facial changes based on the differences in the centroids, the actual movement of facial tissues in the three-dimensional direction can be accurately captured, thereby enabling real-time and accurate determination of facial changes during the beauty process.

[0079] In one exemplary embodiment, such as Figure 3 As shown, another facial image processing method is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps S302 to S314. Wherein:

[0080] Step S302: Obtain the three-dimensional facial model of the target object at different times during the beauty process.

[0081] The 3D facial model can be a structured 3D model of the face based on a triangular mesh, obtained through 3D scanning technology. It is a digital reproduction of the facial morphology of the target object. The triangular mesh is the smallest data unit used to discretize the surface morphology of a 3D geometric object in the 3D facial model, and can be composed of vertices, edges, and triangular faces. Since the face is usually an irregular free-form surface, the discretized triangular mesh can break down complex surfaces into countless simple triangular planes, approximating any concave or convex shape.

[0082] Optionally, a full-area scan of the target object's face can be performed using a 3D scanning device to generate point cloud data. Then, 3D modeling software can be used to convert the point cloud data into a triangular mesh model. Adjacent vertices can be connected through topological reconstruction to form triangular patches covering the entire face, ensuring that the model surface is smooth and consistent with the real facial shape, thus obtaining a 3D facial model of the target object.

[0083] Step S304: Perform 3D alignment on each 3D facial model, and render each 3D facial model uniformly to a preset viewpoint to obtain the 3D facial image data corresponding to each 3D facial model.

[0084] In practice, 3D alignment of the various 3D facial models can eliminate spatial pose differences between models acquired at different times. For example, these differences might manifest as a slight tilt of the head in the 3D facial model at time T0 and a slight tilt of the head in the 3D facial model at time T1. 3D alignment can ensure that key points of the rigid facial structures (such as the tip of the nose, inner corner of the eye, and chin) in each 3D facial model coincide in 3D space. Optionally, the ICP (Iterative Closest Point) algorithm can be used to iteratively calculate the optimal translation and rotation matrices, minimizing the spatial distance between corresponding key points between models, thereby achieving 3D alignment of the various 3D facial models.

[0085] The preset viewpoint can include a fixed viewing angle that is pre-set for uniformly rendering all 3D facial models, such as a frontal view, a side view, or a 45-degree side view. In practice, each 3D facial model can be uniformly rendered to the preset viewpoint based on its viewpoint type (such as frontal or side view) to obtain the corresponding 3D facial image data.

[0086] By performing 3D alignment on 3D facial models of the same target object acquired at different times, it is possible to ensure that subsequent calculations have consistent poses and comparable angles, and to render them uniformly to a predetermined viewpoint.

[0087] Step S306: Identify facial key points in the three-dimensional facial image data.

[0088] Key facial points can include three-dimensional coordinate points on the face that have clear physiological significance and relatively stable positions, used for subsequent facial region division. Examples include the corners of the eyes, the tip of the nose, the ala of the nose, and the midpoint of the root of the nose.

[0089] Optionally, a deep learning-based 3D keypoint detection algorithm can be used to identify facial keypoints in 3D facial image data. The facial keypoints in the 3D facial image data are input into a pre-trained 3D facial keypoint model. The 3D facial keypoint model learns the visual and spatial features of facial anatomy and outputs the 3D coordinates of a preset number of facial keypoints.

[0090] Step S308: Divide the three-dimensional facial image data based on facial key points to obtain multiple facial regions.

[0091] In practice, a fixed reference point R can be selected based on key facial points such as the tip of the nose and the corner of the eye to construct a stable local reference coordinate system. The X, Y, and Z axes are defined as follows: X to the right, Y upward, and Z forward. Point R and its coordinate system are used as the reference base for centroid calculation. Then, based on key facial points such as the corner of the eye, the alar of the nose, and the midpoint of the root of the nose, the left and right cheek regions of the face are divided to obtain multiple facial regions.

[0092] For example, since different facial regions can be defined by different combinations of facial key points, the facial key points corresponding to each facial region can be connected point by point to form a closed irregular polygon, thereby dividing multiple facial regions.

[0093] Step S310: Determine the local facial spatial volume corresponding to the same facial region from multiple facial regions of each three-dimensional facial image data.

[0094] In the specific implementation, any facial region, such as the left cheek region, is determined from multiple facial regions of each three-dimensional facial image data. This facial region is used as the facial region that needs to be compared among the three-dimensional facial image data. Then, the triangular mesh corresponding to this facial region is extracted, and a closed volume is constructed between it and the reference plane in the reference coordinate system to obtain the local facial space volume of this facial region.

[0095] For example, the vertices of each triangular mesh can be obtained, and it can be determined whether each vertex of any triangular mesh is within the three-dimensional boundary of the facial region. If each vertex is within the three-dimensional boundary, the triangular mesh is determined as the triangular mesh corresponding to the facial region. Then, each triangular mesh corresponding to the facial region is extracted and spliced ​​together to obtain the local facial space volume of the facial region.

[0096] In one embodiment, facial mesh data corresponding to the same facial region is extracted from multiple facial regions of the three-dimensional facial image data; a closed geometry is constructed as a local facial space volume based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data.

[0097] Among them, facial mesh data can be a set of triangular meshes with three-dimensional coordinates extracted from a specific facial region of three-dimensional facial image data.

[0098] The reference plane can be a two-dimensional plane determined based on the reference coordinate system of the three-dimensional facial image data, such as a two-dimensional plane parallel to a certain coordinate plane.

[0099] For example, taking the left cheek region as an example, facial mesh data belonging to the left cheek region can be extracted from the 3D facial image data based on the region boundary of the left cheek region. Then, using the reference coordinate system of the 3D facial image data as a reference, a reference plane parallel to a fixed coordinate plane (such as the XZ plane) within the coordinate system can be set. This reference plane can be located at or below the lowest position of the left cheek region and is used to construct the bottom surface of the closed geometry. Then, based on the reference plane, the edge vertices of the facial mesh data in the left cheek region can be extracted. These edge vertices are projected onto the reference plane along the vertical direction of the reference coordinate system to form a bottom contour that perfectly matches the top contour of the left cheek. Then, the bottom contour can be triangulated using the Delaunay Triangulation algorithm to generate facial mesh data covering the range of the bottom contour, which serves as the bottom surface of the closed geometry, ensuring that the bottom surface and the top surface are aligned in the horizontal projection direction. Then, side transition data can be constructed in the spatial gap between the top and bottom surfaces, thereby constructing a closed geometry based on the side transition data, facial mesh data, and the reference plane, which serves as a local facial space volume.

[0100] The technical solution of this embodiment extracts facial mesh data corresponding to the same facial region, which can completely preserve the subtle morphology of the facial region surface. Furthermore, based on the facial mesh data and the reference plane, a closed geometry is constructed, which can transform the facial surface morphology into a three-dimensional entity with complete volume properties. This can be used to calculate key spatial parameters such as volume and centroid, thereby improving the reliability and accuracy of subsequent assessment of facial changes based on centroid.

[0101] In one embodiment, before constructing the closed geometry based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data, the method further includes: determining reference points from the three-dimensional facial image data based on the spatial positional relationship between each facial key point in the three-dimensional facial image data; and constructing a three-dimensional coordinate system of the three-dimensional facial image data as the reference coordinate system using the reference points as the origin of the coordinate system.

[0102] In practice, the spatial relationships between key facial points can include spatial relationships such as distance and angle. For example, the inner corners of the eyes and the tip of the nose form an isosceles triangle, and the chin and the left and right mandibular angles form an inverted triangle, which can be used as constraints for reference point calculation.

[0103] Optionally, for geometric shapes formed by facial key points (such as isosceles triangles), the geometric center of the shape can be calculated as a reference point. Alternatively, different weights can be assigned to different facial key points. For example, since the tip of the nose is located at the center of the face, its weight can be higher than that of the inner corners of the eyes. Then, the weighted average of the coordinates of all facial key points can be calculated as the reference point. For instance, the coordinates of the tip of the nose are P1, with a corresponding weight of w1; the coordinates of the inner corners of the eyes are P2 and P3, with corresponding weights of w2 and w3 respectively; therefore, the coordinates of the reference point can be equal to... .

[0104] Then, a stable local reference coordinate system is constructed based on the reference point, and the X, Y and Z axes are defined as follows: X to the right, Y to the top, and Z to the front. The R point and its coordinate system are used as the reference datum for the centroid calculation.

[0105] The technical solution of this embodiment determines the reference point of the reference coordinate system based on the stable spatial relationship of facial key points, which can ensure the uniformity and stability of the spatial benchmark and provide an accurate spatial quantitative basis for the final facial change assessment.

[0106] Step S312: Obtain the centroid of the local facial space volume corresponding to each three-dimensional facial image data.

[0107] The description of this step can be found in the description of the above embodiments, and will not be repeated here.

[0108] Step S314: Based on the spatial positional changes of each centroid in the reference coordinate system of the three-dimensional facial image data, and the changes in the angles between each centroid and the coordinate axes of the reference coordinate system, determine the facial changes of the same facial region during the beauty process.

[0109] The spatial position change may include the difference in three-dimensional coordinates of the centroid of the local facial space volume corresponding to each three-dimensional facial image data in the reference coordinate system. It can be an index that quantifies the magnitude and direction of the centroid movement. The spatial position change may include the component changes of the X, Y, and Z coordinate axes. , and The change in spatial position can also include the total range of movement, such as the Euclidean distance between two three-dimensional coordinates.

[0110] The change in the coordinate axis angle can include the angle difference between the centroid of the local facial space volume corresponding to each three-dimensional facial image data and a certain coordinate axis of the reference coordinate system. For example, the change in the angle with the Y-axis is an indicator used to quantify the change in the tilt of the centroid direction, reflecting that the facial tissue not only has translation, but may also have local tilt.

[0111] In practical applications, after calculating the geometric centroid coordinates of the local facial spatial volume, the centroid can be transformed to the R reference coordinate system. The three-dimensional spatial coordinates P(x, y, z) of the centroid in the reference coordinate system are recorded, and the angles between the centroid and the X, Y, and Z axes of the reference coordinate system are calculated. .

[0112] As an example, the coordinate differences of each centroid are calculated along the X, Y, and Z axes of the reference coordinate system to obtain the spatial position change components of each coordinate axis. , and The sign of each spatial position change component directly corresponds to the change in the axis direction of the reference coordinate system. For example, A positive value indicates that the centroid moves along the positive Y-axis. At the same time, based on the spatial positional change components of each coordinate axis, the total movement of the centroid in three-dimensional space can be calculated using the Euclidean distance formula, which serves as a reflection of the overall translational scale of the facial region.

[0113] In practice, statistical hypothesis testing methods (such as t-tests) can be used to analyze the spatial position changes or total movement of each coordinate axis to determine the significance of the changes. Specifically, a statistic and the corresponding significance probability (p-value) can be calculated based on the centroid position data at different time points. The calculated p-value is then compared with a preset significance level (such as 0.05 or 0.01). If the p-value is less than the preset significance level, it indicates a statistically significant difference, and the change in spatial position is considered valid. If the p-value is greater than or equal to the preset significance level, the change in spatial position is considered insignificant.

[0114] As another example, starting from the origin of the reference coordinate system, centroid position vectors of local facial spatial volumes corresponding to each 3D facial image data can be constructed. The starting point of the centroid position vector is the origin of the coordinate system, and the ending point is the 3D coordinate of the centroid. The direction of the vector reflects the spatial orientation of the centroid relative to the entire face. Key coordinate axes that are highly relevant to the assessment of facial changes can be selected. For example, the Y-axis can reflect the vertical tilt, and the Z-axis can reflect the front-back tilt. Based on the vector dot product formula, the angle between each centroid vector and the key coordinate axis can be calculated. The size of the angle reflects the degree of deviation of the centroid vector from the axis.

[0115] In practice, statistical hypothesis testing methods (such as t-tests) can be used to analyze the angle differences between the centroids to determine the significance of the directional change. Specifically, a statistic and the corresponding significance probability (p-value) can be calculated based on the angle data at different time points. The calculated p-value is then compared with a preset significance level. If the p-value is less than the preset significance level, it indicates a statistically significant difference, suggesting an effective change in directional tilt. If the p-value is greater than or equal to the preset significance level, it indicates no significant directional adjustment.

[0116] The angle difference between each centroid is compared with the preset angle change threshold. If the angle difference is higher than the angle change threshold, it indicates that an effective directional tilt change has occurred. If the angle difference is less than or equal to the angle change threshold, it indicates that there is no significant directional adjustment.

[0117] In practice, the change in the spatial position of the centroid and the change in the angle between the coordinate axes are used as quantitative indicators to quantify the direction and magnitude of facial lifting, determine whether local facial tissues show an upward lifting or inward concentration trend, and obtain the facial changes. These facial changes can be used to evaluate the immediate or long-term effects of skincare or medical aesthetic treatments.

[0118] For example, for each facial region, after determining that there has been an effective change in the spatial position of the centroid and an effective change in directional tilt, the lifting direction and amplitude can be determined based on the spatial position change components. For instance, the amount of spatial position change in the positive Y-axis direction indicates that an upward lift has occurred with a certain amplitude. Furthermore, the lifting direction can be further refined based on the change in the coordinate axis angle. For example, a decrease in the angle with the Y-axis indicates that the lifting process is closer to the vertical direction, resulting in a higher degree of lifting. Similarly, the amount of spatial position change in the negative X-axis direction indicates that inward tightening has occurred with a certain degree of tightening. Furthermore, the tightening direction can be further refined based on the change in the coordinate axis angle. For example, an increase in the angle with the X-axis indicates that the tightening process is closer to the horizontal direction, resulting in a higher degree of tightening.

[0119] This application embodiment is based on a three-dimensional face model, combined with facial key points and local region division. By analyzing the angular relationship between the volume centroid coordinates of a specific region in the three-dimensional face model and the reference coordinate axis, the changes in facial lifting and tightening are quantified, which can achieve accurate evaluation of lifting and tightening effects based on facial region centroid analysis.

[0120] In another embodiment, such as Figure 4 As shown, a facial image processing method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0121] Step S402: Obtain the three-dimensional facial model of the target object at different times during the beauty process.

[0122] Step S404: Perform 3D alignment on each 3D facial model, and render each 3D facial model uniformly to a preset viewpoint to obtain the 3D facial image data corresponding to each 3D facial model.

[0123] Step S406: Identify facial key points in the three-dimensional facial image data.

[0124] Step S408: Divide the three-dimensional facial image data based on facial key points to obtain multiple facial regions.

[0125] Step S410: Extract facial mesh data corresponding to the same facial region from multiple facial regions of the three-dimensional facial image data.

[0126] Step S412: Based on the spatial relationship between key facial points in the 3D facial image data, determine reference points from the 3D facial image data; use the reference points as the origin of the coordinate system to construct a 3D coordinate system for the 3D facial image data, which serves as the reference coordinate system.

[0127] Step S414: Construct a closed geometry as a local facial space volume based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data.

[0128] Step S416: Obtain the centroid of the local facial space volume corresponding to each three-dimensional facial image data.

[0129] Step S418: Based on the spatial positional changes of each centroid in the reference coordinate system of the three-dimensional facial image data, and the changes in the angles between each centroid and the coordinate axes of the reference coordinate system, determine the facial changes of the same facial region during the beauty process.

[0130] The technical solution of this application provides a precise, objective, and quantifiable three-dimensional facial lifting and tightening assessment method. First, it can realistically reflect three-dimensional structural changes. By modeling and analyzing the centroid of a local closed volume region, it captures the actual movement of facial tissue in three dimensions, avoiding the neglect of depth changes by traditional 2D analysis methods, with an accuracy down to the millimeter level. Second, it can objectively quantify the lifting direction and magnitude of the face. Using the spatial position of the centroid and its angle with the coordinate axes as the assessment basis, it can clearly describe the direction of tissue movement (upward, inward, symmetry improvement, etc.) and can track facial changes during treatment in real time (such as the immediate tightening effect after using skincare products or radiofrequency beauty devices). In addition, it can automatically analyze facial symmetry. By comparing the differences in the centroid positions of the left and right cheek areas, it can quickly determine the symmetry of the facial lifting effect, assisting in the optimization of treatment plans and postoperative adjustments. Finally, it is adaptable to various application scenarios and is suitable for fields such as facial skincare product efficacy verification, medical aesthetic efficacy evaluation, real-time feedback from radiofrequency beauty devices, and pre- and post-operative comparative analysis, providing a standardized quantitative tool for the beauty industry.

[0131] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] For the convenience of those skilled in the art, Figure 5 An exemplary schematic diagram of three-dimensional facial image data is provided. As can be seen, the left image shows an XYZ three-dimensional coordinate system constructed with reference point R as the origin, where point P can be the position of the centroid of the facial region corresponding to the left cheek. The right image shows an XYZ three-dimensional coordinate system constructed with reference point R' as the origin, where P' can be the position of the centroid of the facial region corresponding to the left lower jawline.

[0133] Based on the same inventive concept, this application also provides a facial image processing apparatus for implementing the facial image processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more facial image processing apparatus embodiments provided below can be found in the limitations of the facial image processing method described above, and will not be repeated here.

[0134] In one exemplary embodiment, such as Figure 6 As shown, a facial image processing device is provided, including: a data acquisition module 610, a region determination module 620, a centroid determination module 630, and an effect determination module 640, wherein:

[0135] The data acquisition module 610 is used to acquire multiple three-dimensional facial image data of the target object at different times during the beauty process.

[0136] The region determination module 620 is used to determine local facial spatial volumes corresponding to the same facial region from each of the three-dimensional facial image data.

[0137] The centroid determination module 630 is used to obtain the centroid of the local facial space volume corresponding to each of the three-dimensional facial image data.

[0138] The effect determination module 640 is used to determine the facial changes of the same facial area during the beauty process based on the differences of the centroids.

[0139] In one embodiment, the effect determination module 640 is specifically used to determine the facial changes of the same facial region during the beauty process based on the amount of spatial position change between each centroid in the reference coordinate system of the three-dimensional facial image data, and the amount of angle change between each centroid and the coordinate axis of the reference coordinate system.

[0140] In one embodiment, the region determination module 620 is specifically used to identify facial key points in the three-dimensional facial image data; divide the three-dimensional facial image data based on the facial key points to obtain multiple facial regions; and determine local facial spatial volumes corresponding to the same facial regions from the multiple facial regions of each of the three-dimensional facial image data.

[0141] In one embodiment, the region determination module 620 is specifically used to extract facial mesh data corresponding to the same facial region from the plurality of facial regions of the three-dimensional facial image data; and to construct a closed geometry as the local facial space volume based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data.

[0142] In one embodiment, the region determination module 620 is specifically used to determine a reference point from the three-dimensional facial image data based on the spatial positional relationship between the facial key points in the three-dimensional facial image data; and to construct a three-dimensional coordinate system of the three-dimensional facial image data using the reference point as the origin of the coordinate system, which serves as the reference coordinate system.

[0143] In one embodiment, the data acquisition module 610 is specifically used to acquire three-dimensional facial models of the target object at different times during the beauty process; to perform three-dimensional alignment on each of the three-dimensional facial models; and to render each of the three-dimensional facial models uniformly to a preset viewpoint to obtain three-dimensional facial image data corresponding to each of the three-dimensional facial models.

[0144] Each module in the aforementioned facial image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0145] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a facial image processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0146] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A facial image processing method, characterized in that, The method includes: Acquire multiple 3D facial image data of the target object at different times during the beauty process; Determine the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data; Obtain the centroid of the local facial spatial volume corresponding to each of the three-dimensional facial image data; Based on the differences in the centroids, the facial changes in the same facial area during the beauty process are determined.

2. The method according to claim 1, characterized in that, The step of determining the facial changes in the same facial region during the cosmetic procedure based on the differences in the centroids of each region includes: Based on the spatial positional changes between the centroids in the reference coordinate system of the three-dimensional facial image data, and the changes in the angles between the centroids and the coordinate axes of the reference coordinate system, the facial changes of the same facial region during the beauty process are determined.

3. The method according to claim 1, characterized in that, The step of determining the local facial spatial volume corresponding to the same facial region from each of the three-dimensional facial image data includes: Identify key facial points in the three-dimensional facial image data; The three-dimensional facial image data is divided based on the facial key points to obtain multiple facial regions; From the plurality of facial regions in each of the three-dimensional facial image data, a local facial spatial volume corresponding to the same facial region is determined.

4. The method according to claim 3, characterized in that, Determining the local facial space volume corresponding to the same facial region from the plurality of facial regions in each of the three-dimensional facial image data includes: From the multiple facial regions of the three-dimensional facial image data, extract facial mesh data corresponding to the same facial region; Based on the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data, a closed geometry is constructed as the local facial space volume.

5. The method according to claim 4, characterized in that, Before constructing the closed geometry using the reference plane provided by the reference coordinate system of the facial mesh data and the three-dimensional facial image data, the method further includes: Reference points are determined from the three-dimensional facial image data based on the spatial positional relationship between the key facial points in the three-dimensional facial image data. Using the reference point as the origin of the coordinate system, a three-dimensional coordinate system for the three-dimensional facial image data is constructed, which serves as the reference coordinate system.

6. The method according to claim 1, characterized in that, The acquisition of multiple three-dimensional facial image data of the target object at different times during the beauty process includes: Obtain three-dimensional facial models of the target object at different times during the beauty treatment process; The three-dimensional facial models are aligned in three dimensions, and then rendered uniformly to a preset viewpoint to obtain the three-dimensional facial image data corresponding to each three-dimensional facial model.

7. A facial image processing device, characterized in that, The device includes: The data acquisition module is used to acquire multiple three-dimensional facial image data of the target object at different times during the beauty process; The region determination module is used to determine local facial spatial volumes corresponding to the same facial region from each of the three-dimensional facial image data; The centroid determination module is used to obtain the centroid of the local facial space volume corresponding to each of the three-dimensional facial image data. The effect determination module is used to determine the facial changes of the same facial area during the beauty process based on the differences of the centroids.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images