Recognition and analysis method and system based on facial cosmetic result

Abstract

The invention relates to the technical field of facial recognition and analysis, and discloses a recognition and analysis method and system based on a facial face-lifting result, and the method comprises the steps: carrying out the collection of facial images of a target object before and after the face-lifting; respectively analyzing the completion degree of the expected effect and the preservation degree of the own characteristics to obtain recognition results of the completion degree and the preservation degree; performing feature analysis on each dynamic feature to obtain an evaluation parameter of each image, and performing multi-dimensional action evaluation by using the evaluation parameters to obtain a multi-dimensional action recognition result; and feeding back the multi-dimensional action recognition result to the recognition result of the completion degree to obtain an analysis result after the completion degree is adjusted under the condition of adapting to dynamic characteristics. And finally, a comprehensive analysis result including the face-lifting completion degree, the individual feature retention degree and the dynamic performance adaptability is formed, and the method has the advantages of being high in robustness, high in calculation efficiency and comprehensive in evaluation dimension.

Description

Technical Field

[0001] The present invention relates to the technical field of facial recognition and analysis, and specifically to a recognition and analysis method and system based on the results of facial plastic surgery. Background Art

[0002] In the fields of face recognition, image processing, and biometric analysis, the automatic analysis technology of facial images has been widely used in recent years.

[0003] With the development of computer vision and deep learning algorithms, tasks such as identity recognition, emotion recognition, and expression recognition based on face images have shown significant value in multiple industries such as security, healthcare, beauty, and social networking. Especially in the field of face recognition, technologies such as facial feature extraction, key point localization, and expression change analysis have been continuously matured, laying a solid foundation for intelligent image analysis.

[0004] However, when facial images are subject to external interventions (such as expression movements, angle changes, and lighting changes) or internal changes (such as natural aging, surgical adjustments, etc.), traditional recognition algorithms still face challenges in terms of stability and adaptability.

[0005] In addition, how to take into account individual differences and dynamic expression factors during the image analysis process to ensure recognition accuracy and evaluation credibility has also become an important issue of concern in the industry. Therefore, the research on the comprehensive recognition and analysis method of facial images under multi-dimensional features is particularly crucial. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed.

[0007] Therefore, the technical problems solved by the present invention are: the existing recognition and analysis methods for the results of facial plastic surgery have problems such as overly rigid recognition results; and inability to adapt to the diversification of facial expressions.

[0008] To solve the above technical problems, the present invention provides the following technical solution: A recognition and analysis method based on the results of facial plastic surgery, including:

[0009] Collect facial images of the target object before and after plastic surgery respectively; the facial images include, under different actions, through feature analysis, obtaining the feature markers of each image;

[0010] Classify the feature markers into static features and dynamic features;

[0011] Construct image set A under the static features and image set B under the dynamic features respectively;

[0012] According to the image set A, analyze the completion degree of the expected effect and the preservation degree of its own characteristics respectively, and obtain the recognition results of the completion degree and the preservation degree;

[0013] By analyzing the features of the image set B under each dynamic feature, the evaluation parameters of each image are obtained, and the multi-dimensional action evaluation is performed using the evaluation parameters to obtain the multi-dimensional action recognition result;

[0014] The multi-dimensional action recognition result is fed back into the recognition result of the completion degree to obtain the analysis result after adjustment of the completion degree under the adaptive dynamic feature.

[0015] As a preferred solution of the recognition and analysis method based on the facial plastic surgery result of the present invention, wherein: the feature analysis includes presetting L action categories, and using the pre-trained neural network to use the collected continuous frame images as the input of the neural network;

[0016] Output an information pair including the action category and the confidence level.

[0017] As a preferred solution of the recognition and analysis method based on the facial plastic surgery result of the present invention, wherein: the feature is marked as: for each image, the information pair output by the neural network;

[0018] The static feature includes an information pair with the action category of no expression action; the dynamic feature includes an information pair with the action category of existing expression action;

[0019] Let the index u of the action category represent a positive integer with a maximum value of L; in the image set A, u = L; in the image set B, u ∈ (0, L-1]; the image set A contains subsets: the pre-plastic surgery image subset A+ and the post-plastic surgery image subset A-;

[0020] The image set B contains subsets: the image subsets under each action category, B = {B1, B2,..., B L-1}; where B u represents the subset of the u-th action category;

[0021] In each subset, the image with the highest confidence level is used as the seed image; let f u be the seed image of the u-th action category;

[0022] In the sequence composed of the collected continuous frame images, mark each seed image f u ;

[0023] The growth process of the seed image is specifically as follows:

[0024] Starting from f u and growing frame by frame to both sides in the frame sequence, during the growth process, each frame image is judged. If the action category of the image is the same as that of f uIf the action categories are the same, growth is confirmed; if the action category of the image is different from that of f u If the action categories are different, growth is terminated;

[0025] Seed image f u After growth is completed, evaluate the length of growth of the seed image; if the total length of growth on both sides is less than the preset number of frames, reselect the seed image for the u-th action category. After marking the original seed image as non-selectable, use the image with the highest confidence as the seed image and restart growth until the preset number of frames is satisfied;

[0026] After growth is completed, obtain each seed image and the consecutive frame images it has grown to, as the set of images to be analyzed for each action category; the set of images to be analyzed for the u-th action category is denoted as: AB u ={A0, B u0};

[0027] Among them, if the u-th action category is: no facial expression action occurred, then B u0 is an empty set and A0 = {A0+, A0-}; if the u-th action category is: facial expression action occurred, then A0 is an empty set; A0 represents the set of images to be analyzed for the static features, including the set of images to be analyzed for the static features before plastic surgery A0+ and the set of images to be analyzed for the static features after plastic surgery A0-; B u0 represents the set of images to be analyzed for the u-th action category under the dynamic features;

[0028] When there is no seed image with a total length satisfying the preset number of frames for the u-th action category, select the seed image with the longest growth total length to construct the set of images to be analyzed for the u-th action category.

[0029] As a preferred solution of the recognition and analysis method based on facial plastic surgery results according to the present invention, wherein: the degree of completion includes obtaining a three-dimensional stereoscopic structure model of the target object's face, marking the plastic surgery area, performing a two-dimensional plane projection conversion on the three-dimensional stereoscopic structure model to obtain a two-dimensional image containing the marked plastic surgery area;

[0030] For each frame image in A0-, perform degree of completion recognition;

[0031] Among them, n represents the number of images in A0-; k represents the image index in A0-; C total represents the degree of completion of the plastic surgery area after plastic surgery; max(C k ) represents the maximum value of the similarity between the plastic surgery area of the k-th image in A0- and the marked area in the two-dimensional image at all projection angles;

[0032] The preservation degree includes calculating the completion degree for each frame image in A0+ to obtain C total+ indicating the completion degree of the plastic surgery area before the plastic surgery is completed; and taking the difference between C total and C total+ to obtain the preservation degree P total .

[0033] As a preferred solution of the recognition and analysis method based on the facial plastic surgery result according to the present invention, wherein: the multi-dimensional action evaluation includes, but is not limited to, evaluating the coherence and action amplitude of the image sequence to be analyzed in B u0 to obtain a set of evaluation parameters (G u , Q u ); after performing weighted averaging on the evaluation parameters (G u , Q u ), obtaining comprehensive evaluation parameters (G z , Q Z );

[0034] wherein, G u represents the coherence parameter of B u0 , Q u represents the action amplitude parameter of B u0 , R represents the number of images in B u0 ; r represents the image index in B u0 ; S r，r+1 represents the similarity between the r-th image and the (r + 1)-th image in the sequence of B u0 ; S r,h represents the similarity between the r-th image and the h-th image in the sequence of B u0 ; min represents taking the minimum value; Su0 is the minimum similarity threshold for the u-th action category, indicating the maximum action amplitude.

[0035] As a preferred solution of the recognition and analysis method based on the facial plastic surgery result according to the present invention, wherein: the analysis result after completion degree adjustment is: C ’ total = G z × C total ÷ Q Z .

[0036] As a preferred solution of the recognition and analysis method based on the facial plastic surgery result according to the present invention, wherein: after the recognition is completed, all the results are divided into three parts for display:

[0037] Part one: the recognition results of the completion degree and the preservation degree; Part two: the multi-dimensional action recognition results; Part three: the analysis result after completion degree adjustment.

[0038] An identification and analysis system based on the results of facial plastic surgery using any of the methods described in the present invention, characterized in that:

[0039] An acquisition unit that respectively acquires facial images of a target object before and after plastic surgery.

[0040] An identification unit that analyzes the degree of completion of the expected effect and the preservation degree of its own characteristics respectively according to the image sets under static features before and after plastic surgery, and obtains the identification results of the degree of completion and the preservation degree.

[0041] An analysis unit that obtains the evaluation parameters of each image through the feature analysis of each dynamic feature, and uses the evaluation parameters to perform multi-dimensional action evaluation to obtain multi-dimensional action recognition results.

[0042] An adjustment unit that feeds back the multi-dimensional action recognition results into the recognition results of the degree of completion to obtain the analysis results of the adjusted degree of completion under dynamic features.

[0043] A computer device, comprising: a memory and a processor; the memory stores a computer program, wherein: when the processor executes the computer program, the steps of any of the methods described in the present invention are implemented.

[0044] A computer-readable storage medium, on which a computer program is stored, wherein: when the computer program is executed by a processor, the steps of any of the methods described in the present invention are implemented.

[0045] Advantages of the present invention: The identification and analysis method based on the results of facial plastic surgery provided by the present invention integrates static and dynamic features to perform multi-dimensional identification and analysis of plastic surgery effects, realizing collaborative evaluation at the structural and functional levels. Neural networks are used for action recognition, and the similarity of each frame of image with the expected three-dimensional model at different angles is compared to obtain the results of the degree of completion and the preservation degree; at the same time, action coherence and action amplitude parameters are introduced to construct a dynamic evaluation mechanism, and the static degree of completion results are feedback-corrected. Finally, a comprehensive analysis result including the degree of plastic surgery completion, individual feature retention degree, and dynamic performance adaptability is formed, significantly improving the accuracy, scientificity, and clinical practicability of plastic surgery effect identification. The system has the advantages of strong robustness, high calculation efficiency, and comprehensive evaluation dimensions. Description of the Drawings

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0047] Figure 1 This is the overall flowchart of an identification and analysis method based on facial plastic surgery results provided for the first embodiment of the present invention. Detailed implementation manners

[0048] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will provide a detailed description of the specific implementation manners of the present invention with reference to the accompanying drawings of the specification. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] Example 1, referring to Figure 1 , which is an embodiment of the present invention, provides an identification and analysis method based on facial plastic surgery results, including:

[0050] S1: Collect facial images of the target object before and after plastic surgery respectively; the facial images include, through feature analysis under different actions, obtaining the feature markers of each image.

[0051] Further, use an image acquisition device with high-definition imaging capabilities, such as a digital camera, an industrial camera, a structured light scanner, or a depth camera (such as Kinect, RealSense, etc.) to obtain high-quality facial images. In the solution of this embodiment, images are collected at a total of 7 standard angles including the front, 45-degree left and right sides, 90-degree left and right sides, upward view, and downward view to restore the three-dimensional structural features of the face as comprehensively as possible. In other feasible solutions, other angles of collection can be used.

[0052] Collect the basic images in the static expression state, and at the same time guide the object to complete the preset expression actions (such as smiling, closing eyes, opening mouth, raising eyebrows, etc.) in sequence, and collect continuous frame images in each action state to construct a dynamic feature image sequence. Image acquisition should be carried out under the conditions of uniform illumination and simple background, avoiding interference factors such as shadows and reflections, and ensuring clear images and complete features. The image acquisition before plastic surgery should be completed within one week before the surgery, and the image acquisition after plastic surgery should be carried out after the surgery recovery period (such as more than 1 month) to ensure the stability of the facial structure.

[0053] The feature analysis includes presetting L action categories, and using a pre-trained neural network to take the collected consecutive frame images as the input of the neural network; outputting an information pair including the action category and the confidence. Among them, the neural network classifies the "action category" according to the confidence of each action. Therefore, when outputting, it is inevitable to generate the confidence and the classification result. This solution outputs both of them as an information combination.

[0054] In this embodiment, the neural network structure adopted is: I3D or TCN or CNN-LSTM hybrid structure. Because in this solution, what is needed is not just an ordinary image classification network, but a neural network structure with temporal modeling ability. It is necessary to consider the classification of consecutive frame images before and after. I3D is suitable for action recognition tasks, and the input is a sequence of consecutive frame images; it expands a 2D convolutional network (such as Inception-v1) into a 3D convolutional network, and the convolutional kernel can extract features in the time dimension; it can effectively capture the "difference between the front and back image frames"; output: the classification result of the action category + the confidence of each category (softmax output). The CNN-LSTM network structure includes using CNN (such as ResNet) to extract the spatial features of each frame image in the front; inputting the extracted frame feature sequence into LSTM (long short-term memory network) for temporal modeling; LSTM can model the time context and learn the "relationship between the current frame action and the front and back frames"; output: also the category + the corresponding confidence; suitable for video or frame sequence input, with a simple and flexible structure, suitable for the recognition of a small number of categories of cosmetic actions.

[0055] TCN uses a recurrent neural network and uses one-dimensional temporal convolution to extract the temporal dependencies in the frame sequence; it supports faster parallel training and has stable effects; it is commonly used in action recognition and expression change recognition; it can also output the category + the confidence.

[0056] The feature is marked as: the information pair output by the neural network for each image.

[0057] The feature marks are divided into static features and dynamic features. The static features include the information pairs with the action category of no expression action; the dynamic features include the information pairs with the action category of existing expression actions.

[0058] For example, when "no action" or "static" is recognized by the neural network (such as: calm face, no obvious muscle changes, etc.), the action category in the corresponding output information pair is "no expression action". When actions such as laughing, frowning, blinking, and opening the mouth are recognized by the neural network, the action category in the corresponding output information pair is the corresponding action, classified as "existing expression action".

[0059] It should be noted that in static images, facial expressions do not change, the muscle state is stable, and the contour is clear, which is convenient for extracting structural features such as facial key points and regional contours. Therefore, static features are mainly used to analyze whether the face after plastic surgery achieves the expected morphological effect in a non-movement state, such as nose height, chin contour, facial symmetry, etc., so as to evaluate the completion and retention of plastic surgery. In addition, facial expressions (such as smiling, frowning, and blinking) will cause muscle traction, skin stretching and deformation of facial tissues, which is an important indicator for testing the naturalness and coordination of plastic surgery effects. Dynamic feature images can reflect the performance of plastic surgery areas during actual use, such as whether they are stiff, whether they are asymmetrical, whether there is a break in expression, etc. Therefore, dynamic features are used for multi-dimensional action evaluation, such as coherence, movement amplitude, stability, etc. If classification is not performed, the mixed processing of static and dynamic images will cause the analysis results to be interfered by expression factors, resulting in confusion in feature extraction. For example, the movement amplitude will be misjudged as structural deformation, and the static contour will be distorted due to movement traction, resulting in inaccurate analysis. Therefore, the two types of feature images must be separated and modeled separately to achieve stable and accurate recognition analysis.

[0060] S2: constructing an image set A under the static features and an image set B under the dynamic features respectively.

[0061] Assume that the index u of the action category represents a positive integer with a maximum value of L; in image set A, u=L; in image set B, u∈(0, L-1].

[0062] The image set A includes subsets: a subset A+ of images before plastic surgery and a subset A- of images after plastic surgery.

[0063] The image set B includes subsets: the image subset B under each action category = {B1, B2, ..., B L-1}.

[0064] Among them, B u represents the subset of the u-th action category. In the image set B, u∈(0, L-1] (one of the L categories is a "static feature without action"); if the subset of action category B u Empty Set It means that the action category cannot be identified and is marked as Indicates an empty set. If an empty set appears, it means that the expression is missing, be alert, and issue a warning separately. When , the “multi-dimensional action evaluation” in the following text does not measure the empty set part.

[0065] Furthermore, in each subset, the image with the highest confidence is used as the seed image; let f u is the seed image of the u-th action category.

[0066] In the sequence composed of the collected consecutive frame images, each seed image f is marked u .

[0067] The growth process of the seed image is specifically as follows:

[0068] Starting from f u , it grows frame by frame to both sides in the frame sequence. During the growth process, each frame image is judged. If the action category of the image is the same as that of f u , the growth is confirmed; if the action category of the image is different from that of f u , the growth is terminated.

[0069] After the seed image f u completes growth, evaluate the length of the growth of the seed image; if the total length of the growth to both sides is less than the preset number of frames, reselect the seed image of the u-th action category. After marking the original seed image as unselectable, use the image with the highest confidence as the seed image and grow again until the preset number of frames is satisfied. If the number of frames grown on both sides of the seed image is too small, it may only cover the starting or ending stage of the action and cannot fully reflect the complete change process of the action, resulting in distorted analysis results. Therefore, by setting the minimum growth length, it is ensured that the extracted image sequence can cover the main stages of the entire action process. By dynamically reselecting the new image with the highest confidence as the seed and restricting the original seed from being selected again, automatic iteration and optimization of the selection of the seed image are realized, and finally an image sequence with the most representative and good continuity under the action category is obtained.

[0070] After completion of growth, obtain each seed image and the consecutive frame images it grows to as the set of image sequences to be analyzed for each action category; the set of image sequences to be analyzed for the u-th action category is denoted as: AB u ={A0, B u0}

[0071] Among them, if the u-th action category is: no expression action occurred, then B u0 is an empty set and A0 = {A0+, A0-}; if the u-th action category is: expression action occurred, then A0 is an empty set; A0 represents the set of image sequences to be analyzed for the static features, including the set of image sequences to be analyzed for the static features before plastic surgery A0+ and the set of image sequences to be analyzed for the static features after plastic surgery A0-; B u0 represents the set of image sequences to be analyzed for the u-th action category under the dynamic features (one of the u action categories is a static category, so this serial number can only be selected up to the L-1th position, and the Lth action category is "no expression action occurred").

[0072] It should be noted that there is only one A0. Therefore, when designing, A0 is generally not set as the nth action category, but directly set as the Lth. This will not affect the sequence order of u in B (B u0 is multiple, and there are multiple sets according to different u).

[0073] When there is no seed image with a total length satisfying the preset number of frames for the u-th action category, select the seed image with the longest growth total length to construct the set of image sequences to be analyzed under the u-th action category.

[0074] It should be noted that when performing recognition and analysis on each action category, it is necessary to extract the image segments representing the action category from the continuous image sequence. Since the entire video or image sequence contains a large number of redundant frames, if processed without screening, it will lead to an increase in the system's computational burden, and some frames may contain noise or atypical actions, affecting the recognition accuracy. This solution can find a continuous frame segment that is most representative of each expression by selecting seeds, which can reduce the computational amount and be accurate for analysis.

[0075] Seed images are often in the "high-expression region" of a certain action category, that is, the frames with the clearest and most typical expressions. Growing around this as the center can accurately capture a period of time segment with the most expressive power of the action, avoiding misjudgment caused by using the entire video or low-quality frames. Instead of analyzing the entire image sequence, only focus on the small segment of high-quality continuous frames obtained by growing from the seed image, which greatly reduces the computational burden in links such as feature extraction, similarity calculation, and coherence evaluation, and improves the overall processing efficiency. Since the image with the highest confidence has greater classification clarity and boundary clarity, the image sequence constructed with this as the core is also more stable and representative in terms of feature coherence, action amplitude, etc., which helps to improve the accuracy and credibility of subsequent analysis (such as completion adjustment).

[0076] S3: According to the image set A, analyze the completion degree of the expected effect and the preservation degree of its own characteristics respectively to obtain the recognition results of the completion degree and the preservation degree.

[0077] The completion degree includes obtaining a three-dimensional stereoscopic structure model of the target object's face, marking the plastic surgery area, and performing two-dimensional plane projection conversion on the three-dimensional stereoscopic structure model to obtain a two-dimensional image containing the plastic surgery area mark.

[0078] It should be noted that during the actual plastic surgery plan design phase, doctors or designers usually use professional 3D modeling software (such as FaceGen, 3D-Me, ZBrush, or dedicated simulation systems in the medical industry) to preset the facial shape after plastic surgery for the client. This 3D modeling process is based on the facial scan data of the target object, combined with the plastic surgery intention, and generates a 3D model of the changed face by adjusting parameters. This model is not only used for preoperative simulation and client preview but also serves as the source of the "target expected form" in the present invention.

[0079] Therefore, a 3D solid structure model is established before plastic surgery, with complete facial structure feature data, serving as the target model for subsequent completion degree comparative analysis. During the modeling process, professional doctors or designers select the areas involved in plastic surgery on the 3D model surface and label them as a set of area tags on the facial grid (such as: nasal area, mandibular area, cheek areas, etc.). Or mark them automatically: use face analysis tools (such as dlib, FaceMesh, OpenFace) to automatically detect and divide the key facial areas, and then automatically check the areas involved in plastic surgery operations as the "marked areas" through system settings.

[0080] Among them. The purpose of 2D projection conversion is to project the 3D structure onto a standard 2D image for matching and comparison with the actually captured image. Select a standard viewing angle (such as the front view, 45° from the left, 45° from the right, etc.), set the pose matrix (rotation + translation) of the virtual camera; use the perspective projection model or the orthographic projection model to convert the 3D grid points (X, Y, Z) into 2D image coordinates (x, y); the marked points in the plastic surgery area are projected onto the 2D image simultaneously, forming a 2D image "containing plastic surgery area marks", which can be used for subsequent feature comparison with the corresponding area in the actual image.

[0081] For each frame image in A0-, perform completion degree recognition;

[0082] Among them, n represents the number of images in A0-; k represents the image index in A0-; C total represents the completion degree of the plastic surgery area after plastic surgery; max(C k ) represents the maximum similarity of the plastic surgery area in the k-th image in A0- and the marked area in the 2D image under all projection angles.

[0083] It should be noted that each frame of the actually captured images has a different angle. If directly compared with a target image at a fixed angle, errors are likely to occur. Therefore, it is designed to select the angle that best matches the current frame from multiple projection angles for comparison, which is more adaptable and objective. Dynamically selecting the optimal-angle projection map as a reference helps identify the visual state that truly expresses the plastic surgery effect and avoids low scores caused by angle errors. Independent scoring for each frame can be used for subsequent statistical analysis, trend judgment, or average completion degree evaluation, realizing fine-grained dynamic plastic surgery analysis.

[0084] Among them, for the similarity, SSIM can be used, or the cosine similarity of the feature vectors of the plastic surgery area can be calculated based on deep features (such as FaceNet, ArcFace).

[0085] The preservation degree includes calculating the completion degree for each frame image in A0+ to obtain C total+ indicating the completion degree of the plastic surgery area before the plastic surgery is completed (the calculation method is similar to C total , and only the parameters need to be adaptively replaced); taking the difference between C total and C total+ to obtain the preservation degree P total . In traditional methods, only the differences before and after are compared, but this kind of comparison cannot achieve an all-round comparison effect. Because for the images collected before and after plastic surgery, the "completion degree calculation result" of the target actually analyzes the actual similarity between the facial area before plastic surgery and the target object. The above calculation has a more comprehensive analysis result. Then taking the difference between the two "completion degrees" before and after plastic surgery can obtain the most comprehensive result of "feature retention". If only the analysis between different frames before and after plastic surgery is carried out, analysis deviations may occur due to problems such as angles. Therefore, by using the target object as a transition, it has high robustness. Or put it another way: "Except for what has been done, it's what hasn't been done." All in this way, the measurement of the "preservation degree" can be achieved most quickly and effectively.

[0086] S4: Through the feature analysis of the image set B under each dynamic feature, obtain the evaluation parameters of each image, and use the evaluation parameters to perform multi-dimensional action evaluation to obtain multi-dimensional action recognition results.

[0087] The multi-dimensional action evaluation includes, but is not limited to, in B u0 , evaluating the coherence and action amplitude of the image sequence to be analyzed to obtain a set of evaluation parameters (G u , Q u ).

[0088] In other alternative embodiments, it is also possible to analyze the degree of adaptation of the entire face, and use a neural network to analyze whether there is a sense of incongruity in the entire face after plastic surgery. Or analyze the degree of organ distortion to obtain whether there are exaggerated movements when the face makes movements.

[0089] Among them, G u represents the coherence parameter, Q u represents the movement amplitude parameter. The larger its value, the smaller the movement amplitude that can be performed, indicating that the restriction on the face after plastic surgery is greater. At the same time, since the sequence for evaluation is obtained by selecting seed nodes, the most characteristic movement, so in fact the movement amplitude should be the maximum amplitude under this expression (which can also be understood as the degree of freedom of facial movement). R represents the number of images in B u0 ; r represents the image index in B u0 ; S r，r+1 represents the similarity between the r-th image and the (r + 1)-th image in the sequence of B u0 ; S r,h represents the similarity between the r-th image and the h-th image in the sequence of B u0 ; min represents taking the minimum value; Su0 is the minimum similarity threshold for the u-th action category, indicating the maximum movement amplitude.

[0090] After performing a weighted average on the evaluation parameters (G u , Q u ), the comprehensive evaluation parameters (G z , Q Z ) are obtained.

[0091] It should be noted that the process of weighted average is achieved by dividing each action coherence parameter (or action amplitude parameter) by its standard value.

[0092]

[0093] Among them, G u0 represents the standard value of the coherence parameter of B u0 , Q u0 represents the standard value of the action amplitude parameter of B u0 . L-1 represents the number of seeds under dynamic features.

[0094] S5: Feed the multi-dimensional action recognition results back into the recognition results of the completion degree to obtain the analysis results after adjustment of the completion degree under the adaptive dynamic features.

[0095] The analysis result after adjustment of the completion degree is: C ’ total = Gz ×C total ÷Q Z 。

[0096] In the actual evaluation of facial plastic surgery effects, it is difficult to comprehensively reflect the performance of plastic surgery effects during actual use only by relying on the recognition of the completion degree under static images. Since the face is in a frequent movement state in daily life (such as smiling, frowning, speaking, etc.), the performance of the plastic surgery area under dynamic conditions (whether it is natural, symmetrical, and stable) will directly affect the overall evaluation of plastic surgery. By using dynamic feature feedback such as coherence parameters and action amplitude parameters to correct the static completion degree, the final analysis result not only retains the evaluation at the structural level but also reflects the functional performance, with higher credibility and practical value. The result after completion degree adjustment can help doctors or designers identify potential problem areas in the dynamic state.

[0097] After the recognition is completed, all results are presented in three parts:

[0098] Part 1: The recognition results of the completion degree and the preservation degree; Part 2: The recognition results of multi-dimensional actions; Part 3: The analysis results after the completion degree is adjusted.

[0099] On the other hand, this embodiment also provides an identification and analysis system based on facial plastic surgery results, which includes:

[0100] An acquisition unit that respectively acquires facial images of the target object before and after plastic surgery.

[0101] An identification unit that respectively analyzes the completion degree of the expected effect and the preservation degree of its own characteristics according to the image sets under static features before and after plastic surgery, and obtains the recognition results of the completion degree and the preservation degree.

[0102] An analysis unit that obtains the evaluation parameters of each image through feature analysis of each dynamic feature, and uses the evaluation parameters for multi-dimensional action evaluation to obtain multi-dimensional action recognition results.

[0103] An adjustment unit that feeds back the multi-dimensional action recognition results into the recognition results of the completion degree to obtain the analysis results after the completion degree is adjusted under dynamic features.

[0104] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs, Read-Only Memories), random access memories (RAMs, Random Access Memories), magnetic disks, or optical discs that can store program codes.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0106] More specific examples (nonexhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), fiber optic devices, and portable compact disc read-only memories (CDROMs). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0107] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0108] Embodiment 2, an embodiment of the present invention, provides a recognition and analysis method based on facial plastic surgery results. To verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0109] Six volunteer participants (numbered P1 - P6) who have completed facial plastic surgery are invited as test subjects. The plastic surgery projects cover common parts, including rhinoplasty, mandibular angle trimming, double eyelid formation, and facial filling, etc. The surgeries have all been completed and the recovery period exceeds 1 month, with stable facial conditions, meeting the conditions for carrying out objective recognition and subjective evaluation.

[0110] The FaceGen modeling tool is used to generate personalized three-dimensional expected models, and the standard pre-plastic surgery styling process is used to confirm the plastic surgery goals. The image data before and after plastic surgery is collected by an Intel RealSense D435i depth camera. The collection angles include 7 perspectives such as the front, 45 degrees left / right, 90 degrees side view, upward view, and downward view. At the same time, continuous frame image sequences are recorded under 5 types of dynamic expressions such as "smiling, opening the mouth, raising the eyebrows, frowning, and closing the eyes", with each expression lasting for 1.5 seconds and a frame rate of 30 fps, and a total of 1050 frames of images are collected per subject.

[0111] The facial feature vectors of each frame of image are extracted through the ArcFace neural network, and the I3D model is used for action classification to output the expression category and the corresponding confidence level. According to the preset rules, the images are divided into a static feature set and a dynamic feature set, and image sets A (static) and B (dynamic) are respectively established. Under each type of action, seed images are selected based on the confidence level, and an action coherence and amplitude determination mechanism is used to automatically grow the image sequence to construct a subset of images to be analyzed.

[0112] The completion degree is calculated by extracting FaceNet embedding features and comparing them using cosine similarity on the two-dimensional images projected by the target model. The dynamic feature index evaluates the action coherence and naturalness based on the mean and variation range of cosine similarity between adjacent frames. In the final stage, the dynamic results are fed back and fused into the completion degree score to form the "adjusted completion degree index" to measure the adaptability and expressiveness of the actual plastic surgery effect in the dynamic real scenario.

[0113] In the experimental completion stage, plastic surgeons were organized to fill in the "Satisfaction Scale for Recognition Results" to subjectively evaluate whether the provided recognition and analysis results truly reflected the plastic surgery effect and dynamic performance. A 5-point scale was used (1 being extremely dissatisfied and 5 being extremely satisfied), and some of the data are recorded in Table 1.

[0114] Table 1 Data Record Sheet

[0115] Parameter Name / Number P1 P2 P3 P4 P5 P6 Static Completion Degree (0 - 1) 0.86 0.83 0.91 0.84 0.88 0.92 Action Coherence Parameter (0 - 1) 0.91 0.89 0.95 0.88 0.90 0.96 Action Amplitude Parameter (the larger the value, the more restricted) 1.30 2.00 0.50 1.70 1.10 0.55 Adjusted Completion Degree Score (automatically calculated) 0.60 0.37 1.73 0.44 0.72 1.61 Preservation Degree Score (0 - 1) 0.81 0.77 0.85 0.78 0.82 0.88 Doctor's Score - Structural Restoration Degree (1 - 5) 5 4.5 5 4.5 5 5 Doctor's Score - Dynamic Naturalness (1 - 5) 5 4.5 5 4.5 5 5 Doctor's Score - Overall Evaluation of Credibility (1 - 5) 5 5 5 5 5 5

[0116] It can be seen that the average static completion degree is 0.89, indicating that the system has good accuracy in multi-view structure restoration. The scores of P3 and P6 are as high as 0.91 and 0.92, showing that the system is very accurate in identifying geometric changes in fine-structured plastic surgery areas such as the nose wings and mandible.

[0117] The average dynamic coherence reaches 0.915, reflecting the high stability of the system in seed image growth and continuous frame action sequence extraction. Combining with the multi-dimensional comparison structure of the present invention, this parameter successfully captures the temporal continuity in natural actions, ensuring that subsequent analysis is carried out on a natural basis.

[0118] The action amplitude parameter reflects the "degree of freedom" of facial actions. The system automatically calculates the action amplitude through the variation range of key points between frames. If the amplitude value is large (such as 2.00 for P2 and 1.70 for P4), it indicates that the action amplitude is restricted to a certain extent after plastic surgery; while a small amplitude value (such as 0.50 for P3 and 0.55 for P6) indicates a high degree of freedom of movement in that area and that the plastic surgery does not pose a restriction. The system will accordingly moderately reduce or enhance the completion degree score to make it more in line with the actual usage state.

[0119] For example, the adjusted completion degree score of P3 is 1.73, much higher than the static score of 0.91, indicating that the system recognizes its natural expression and complete action and gives a positive feedback; while for P2, due to the restricted amplitude (2.00), even though the static completion degree is relatively high (0.83), the adjusted score is reduced to 0.37, prompting the doctor to pay attention to the risk of "good structure but restricted expression".

[0120] This dynamic feedback mechanism is a dimension fusion ability that is difficult to achieve by traditional means, reflecting the technical novelty of this method.

[0121] From the doctor's scores, the scores of all subjects on the structural restoration degree, dynamic naturalness, and overall credibility were all above 4.5 points. Among them, the average value of dynamic naturalness was 4.83, and the credibility was a full score of 5.00, indicating that the system recognition results were not only consistent with the objective parameters but also highly consistent with the doctors' clinical observations.

[0122] It is particularly worth mentioning that in amplitude-limited samples such as P2 and P4, doctors confirmed the presence of slightly restricted facial expressions, and the system scores simultaneously reflected this difference; in amplitude-sufficient samples such as P3 and P6, the system gave extremely high scores, and doctors also stated that "the facial expressions are natural and there is no stiffness". This double consistency of system recognition + clinical verification is the most valuable part of the present invention, indicating that this method not only has data-driven accuracy but also has the ability to explain medical reality problems.

[0123] In summary, through the dynamic adjustment mechanism, the present invention identifies the degree of limitation of the facial dynamic expression ability in different plastic surgery areas, does not rely on a single judgment of the static structure, and constructs an evaluation system that highly adapts to the actual plastic surgery results, with obvious creativity and practicality.

[0124] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A recognition and analysis method based on the results of facial plastic surgery, characterized in that, Including: Collect facial images of the target object before and after plastic surgery respectively; The facial images include, through feature analysis under different actions, obtaining the feature markers of each image; Dividing the feature markers into static features and dynamic features; Constructing the image set A under the static features and the image set B under the dynamic features respectively; According to the image set A, analyzing the completion degree of the expected effect and the preservation degree of its own characteristics respectively, and obtaining the recognition results of the completion degree and the preservation degree; Through the feature analysis of the image set B under each dynamic feature, obtaining the evaluation parameters of each image, and using the evaluation parameters to perform multi-dimensional action evaluation to obtain multi-dimensional action recognition results; Feeding back the multi-dimensional action recognition results into the recognition results of the completion degree to obtain the analysis results after the completion degree is adjusted under the adaptation of dynamic features.

2. The recognition and analysis method based on facial plastic surgery results according to claim 1, wherein: The feature analysis includes presetting L action categories, and using the continuously captured frame images as the input of the neural network through pre-training; Outputting an information pair containing the action category and the confidence level.

3. The recognition and analysis method based on the facial plastic surgery result according to claim 2, wherein: The feature marker is: for each image, the information pair output by the neural network; The static features include information pairs with the action category of no expression action; the dynamic features include information pairs with the action category of having expression actions; Let the index u of the action category represent a positive integer with a maximum value of L; in the image set A, u = L; in the image set B, u ∈ (0, L - 1]; the image set A contains subsets: the pre-plastic surgery image subset A+ and the post-plastic surgery image subset A-; The image set B contains subsets: the image subsets under each action category, B = {B1, B2,..., B L-1}; where B u represents the subset of the u-th action category; In each subset, the image with the highest confidence is used as the seed image; let f u be the seed image for the u-th action category; In the sequence composed of the collected consecutive frame images, mark each seed image f u ; The specific growth process of the seed image is as follows: Starting from f u Beginning, the frame sequence grows frame by frame towards both sides. During the growth process, each frame of the image is judged. If the action category of the image is the same as that of f u , then the growth is confirmed; if the action category of the image is different from that of f u , then the growth is terminated; Seed image f u After the growth is completed, evaluate the length of the growth of the seed image; if the total length of the growth on both sides is less than the preset number of frames, reselect the seed image of the u-th action category. After marking the original seed image as unselectable, use the image with the highest confidence as the seed image and perform growth again until the preset number of frames is satisfied; After the growth is completed, each seed image and the consecutive frame images it grows into are obtained as the set of image sequences to be analyzed for each action category; the set of image sequences to be analyzed for the $u$-th action category is denoted as: AB u = {A0, B u0}; Among them, if the u-th action category is: no facial expression action occurs, then B u0 is an empty set and A0 = {A0+, A0-}; if the u-th action category is: facial expression action occurs, then A0 is an empty set; A0 represents the set of image sequences to be analyzed of the static features, including the set of image sequences A0+ to be analyzed of the static features before plastic surgery and the set of image sequences A0- to be analyzed of the static features after plastic surgery; B u0 represents the set of image sequences to be analyzed of the u-th action category under the dynamic features; When there is no seed image with a total length satisfying the preset number of frames for the u-th action category, select the seed image with the longest growth total length to construct the set of image sequences to be analyzed under the u-th action category.

4. The recognition and analysis method based on facial plastic surgery results according to claim 3, wherein: The completion degree includes obtaining a three-dimensional stereo structure model of the target object's face, marking the plastic surgery area, and performing two-dimensional plane projection conversion on the three-dimensional stereo structure model to obtain a two-dimensional image containing the plastic surgery area mark; For each frame image in A0-, perform completion degree recognition; Among them, n represents the number of images in A0-; k represents the image index in A0-; C total represents the completion degree of the plastic surgery area after plastic surgery; max(C k ) represents the plastic surgery area of the k-th image in A0-, and the similarity maximum value with the marked area in the two-dimensional image under all projection angles; The preservation degree includes calculating the completion degree for each frame image in A0+ to obtain C total+ indicating the completion degree of the plastic surgery area before the plastic surgery is completed; taking C total and subtracting it from C total+ to obtain the preservation degree P total .

5. The recognition and analysis method based on the facial plastic surgery result according to claim 4, wherein: The multi-dimensional motion evaluation includes, but is not limited to, in B u0 , evaluating the coherence and motion amplitude of the image sequence to be analyzed to obtain a set of evaluation parameters (G u , Q u ); after performing weighted averaging on the evaluation parameters (G u , Q u ), obtaining comprehensive evaluation parameters (G z , Q Z ); Among them, G u represents the coherence parameter of B u0 ; Q u represents the motion amplitude parameter of B u0 ; R represents the number of images in B u0 ; r represents the image index in B u0 ; S r，r+1 represents the similarity between the r-th image and the (r + 1)-th image in the sequence of B u0 ; S r,h represents the similarity between the r-th image and the h-th image in the sequence of B u0 ; min represents taking the minimum value; Su0 is the minimum similarity threshold for the u-th action category, representing the maximum motion amplitude.

6. The recognition and analysis method based on the facial plastic surgery result according to claim 5, wherein: The analysis result after completion degree adjustment is: C ’ total = G z × C total ÷ Q Z .

7. The recognition and analysis method based on facial plastic surgery results according to claim 6, characterized in that: After the recognition is completed, all the results are displayed in three parts: Part one: The recognition results of the completion degree and the preservation degree; Part two: The multi-dimensional action recognition results; Part three: The analysis results after the completion degree is adjusted.

8. An identification and analysis system based on facial plastic surgery results using the method according to any one of claims 1-7, characterized in that: A collection unit that respectively collects facial images of the target object before and after plastic surgery; An identification unit that respectively analyzes the completion degree of the expected effect and the preservation degree of its own characteristics according to the image sets under the static features before and after plastic surgery, and obtains the recognition results of the completion degree and the preservation degree; An analysis unit that obtains the evaluation parameters of each image through the feature analysis of each dynamic feature, and uses the evaluation parameters to perform multi-dimensional action evaluation to obtain multi-dimensional action recognition results; An adjustment unit feeds the multi-dimensional action recognition result back into the recognition result of the completion degree, and obtains an analysis result with the completion degree adjusted under dynamic features.

9. A computer device, comprising: A memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images