Facial expression muscle dynamic coordination evaluation system based on deep learning

Abstract

The application discloses a kind of facial expression muscle dynamic coordination evaluation system based on deep learning, including data acquisition module, for collecting the multi-object high frame rate facial dynamic video of preoperative baseline period of subject and the multi-object high frame rate facial dynamic video of postoperative review period;Data preprocessing module is used to generate preoperative three-dimensional facial coordinate dataset and postoperative three-dimensional facial coordinate dataset;Vector field extraction module, extract preoperative full-face expression muscle three-dimensional displacement vector field and postoperative full-face expression muscle three-dimensional displacement vector field;Feature extraction module, extract the physiological asymmetry reference feature set of bilateral corresponding muscle area before operation, and the motion feature set of bilateral corresponding muscle area after operation;Model inference module, output eliminates the quantitative value of diagnosis and treatment related muscle movement symmetry deviation and dynamic coordination degree change quantitative value after innate physiological asymmetry;Result generation module generates the expression muscle dynamic coordination change evaluation result of corresponding subject before and after oral diagnosis and treatment.

Description

Technical Field

[0001] This invention relates to the fields of oral medicine, computer vision, deep learning and medical testing technology, specifically to a deep learning-based dynamic collaborative evaluation system for facial expression muscles. Background Technology

[0002] Orthodontics and prosthodontics are core areas of clinical diagnosis and treatment in oral medicine. Precise evaluation of facial aesthetics and muscle function after treatment is a crucial aspect of clinical protocol development, efficacy assessment, and postoperative rehabilitation guidance. With the rapid development of digital oral medicine, machine vision-based facial motion quantitative evaluation technology has become an important development direction for replacing traditional subjective human evaluation and improving the accuracy of diagnosis and treatment assessment.

[0003] Existing quantitative evaluation techniques for facial muscle movements mostly employ high-frame-rate visual acquisition devices to capture dynamic facial videos of subjects. They then extract facial muscle displacement features using feature detection algorithms and combine this with deep learning algorithms to analyze the symmetry of facial muscle movement on both sides, providing quantitative evaluation indicators for facial aesthetic outcomes after oral treatment. The core technical approach of existing solutions involves establishing a universal standard template for healthy individuals or using large-sample mean data of healthy individuals as a sole evaluation benchmark. By comparing the muscle movement characteristics of the subject with this universal benchmark, the corresponding symmetry evaluation results are output. This technology has already been preliminarily applied in static facial aesthetic evaluation scenarios.

[0004] The core flaw of existing technologies lies in their use of general standard templates or the average of healthy individuals as the sole evaluation benchmark. They fail to consider the baseline characteristics of the subject's facial muscle movements before surgery, cannot distinguish between the muscle movement deviations newly added after orthodontic or restorative procedures and the subject's innate physiological facial asymmetry, and cannot accurately locate the changes in facial expression muscle movement function directly caused by the treatment procedure. As a result, the evaluation results cannot truly reflect the actual clinical effect of the treatment plan and cannot provide accurate and reliable digital basis for optimizing and adjusting clinical treatment plans. Summary of the Invention

[0005] The present invention aims to at least partially solve the technical problems in the above-mentioned technologies.

[0006] Therefore, this invention discloses a deep learning-based facial expression muscle dynamic collaborative evaluation system, characterized in that it includes:

[0007] The data acquisition module is used to acquire multi-view high frame rate facial dynamic videos of the subjects during the preoperative baseline period and multi-view high frame rate facial dynamic videos of the same subjects during the postoperative follow-up period. The video acquisition content during the preoperative baseline period and the postoperative follow-up period is the same standardized oral diagnosis and treatment-specific facial expression and movement sequence.

[0008] The data preprocessing module is used to perform three-dimensional spatial calibration on the multi-view high frame rate facial dynamic video during the preoperative baseline period and the multi-view high frame rate facial dynamic video during the postoperative follow-up period, and generate a preoperative three-dimensional facial coordinate dataset and a postoperative three-dimensional facial coordinate dataset, respectively.

[0009] The vector field extraction module, based on the dense optical flow detection algorithm, performs temporal frame processing on the preoperative three-dimensional facial coordinate dataset and the postoperative three-dimensional facial coordinate dataset to extract the preoperative three-dimensional displacement vector field of the facial muscles and the postoperative three-dimensional displacement vector field of the facial muscles.

[0010] The feature extraction module, based on prior knowledge of oral and maxillofacial muscle anatomy, divides the target muscle regions of the three-dimensional displacement vector field of the preoperative and postoperative three-dimensional displacement vector fields of the facial muscles, and extracts the physiological asymmetry benchmark feature set of the corresponding muscle regions on both sides before surgery, and the motion feature set of the corresponding muscle regions on both sides after surgery.

[0011] The model inference module is used to input the preoperative physiological asymmetry baseline feature set and the postoperative motor feature set into a pre-trained Siamese network-based personalized evaluation model. The Siamese network-based personalized evaluation model is equipped with a feature difference layer. Through the feature difference layer, the input preoperative baseline features and postoperative motor features are differentially calculated, and feature components that match the preoperative physiological asymmetry are removed. Only the muscle movement change components caused by the diagnosis and treatment operation are retained. Based on the retained feature components, the quantitative values ​​of diagnosis and treatment-related muscle movement symmetry deviation and dynamic coordination degree change are output.

[0012] The results generation module generates evaluation results of the dynamic synergistic changes of facial muscles before and after oral treatment for the corresponding subjects, based on the quantitative values ​​of the symmetry deviation of the treatment-related muscle movements and the quantitative values ​​of the dynamic synergistic changes.

[0013] The deep learning-based facial expression muscle dynamic collaborative evaluation system disclosed in this invention can eliminate the interference of subjects' innate physiological facial asymmetry, accurately quantify the changes in facial expression muscle motor function caused by oral diagnosis and treatment, improve the accuracy of efficacy evaluation, and provide a reliable digital basis for optimizing clinical diagnosis and treatment plans.

[0014] In addition, the deep learning-based facial expression muscle dynamic collaborative evaluation system disclosed in this invention may also have the following additional technical features:

[0015] Furthermore, the data acquisition module includes a multi-view acquisition unit. During the preoperative baseline period and the postoperative follow-up period, the subject is guided by a visual guidance interface to complete the standardized oral treatment-specific facial expression and movement sequence according to a preset time sequence. The multi-view acquisition unit simultaneously acquires multi-angle facial dynamic videos.

[0016] Furthermore, the data preprocessing module performs single-target and stereo calibration on the multi-camera acquisition unit using a checkerboard calibration board, calculating the intrinsic parameters and distortion coefficients of each camera, as well as the extrinsic parameters between the multi-cameras. Based on the camera intrinsic parameters and distortion coefficients, distortion correction is performed on each frame of the acquired video to obtain a distortion-free frame sequence. Based on the extrinsic parameters of the multi-cameras, stereo correction and epipolar alignment are performed on the distortion-free frame sequence to generate a pixel-level aligned multi-view frame sequence. The multi-view frame sequence is then used for 3D reconstruction to generate a corresponding 3D facial coordinate dataset.

[0017] Furthermore, the dense optical flow detection algorithm of the vector field extraction module performs pixel matching on the continuous temporal frames of the preoperative 3D facial coordinate dataset and the postoperative 3D facial coordinate dataset. Using the pre-extracted key facial feature points as rigid constraints, it calculates the 3D displacement vector of each facial pixel in each frame. All 3D displacement vectors are smoothed by temporal moving average, and then Gaussian filtering is used to remove high-frequency noise and eliminate displacement interference caused by non-muscle movement, generating a temporally continuous 3D displacement vector field.

[0018] Furthermore, the feature extraction module is used to pre-establish a facial muscle region mapping template related to orthodontic and restorative treatment based on prior knowledge of oral and maxillofacial muscle anatomy. The mapping template includes five bilateral muscle regions with the same name: orbicularis oris, zygomaticus major, zygomaticus minor, levator labii superioris, and masseter. It also includes the facial pixel coordinate mapping relationship corresponding to each muscle region. Based on the facial muscle region mapping template, the target muscle region is divided, a one-to-one correspondence between bilateral muscle regions with the same name is established, and the three-dimensional displacement vector data corresponding to each divided region is extracted.

[0019] Furthermore, the feature extraction module is used to perform temporal alignment of the three-dimensional displacement vector data of the bilateral corresponding muscle regions in the preoperative full-face facial expression muscle three-dimensional displacement vector field, and to match the key action nodes of the standardized oral treatment-specific facial expression action sequence; calculate the statistical mean and distribution characteristics of the difference in motion amplitude, the difference in motion phase, and the deviation in motion trajectory of the bilateral corresponding muscle regions under each key action node, combine the statistical results corresponding to all key action nodes to generate the preoperative physiological asymmetry benchmark feature set, and use the same logic to generate the postoperative motion feature set.

[0020] Furthermore, the model inference module is used to pre-build the personalized evaluation model based on Siamese networks. The model includes two temporal feature extraction branches with completely shared weights, a feature difference layer, and a regression output layer. The temporal feature extraction branch adopts a 3-layer LSTM structure to extract the temporal dependency features of the input features. The feature difference layer is used to calculate the difference between the high-dimensional feature vectors output by the two branches. The model is pre-trained using paired pre- and post-operative facial expression muscle movement annotation datasets, and the model parameters are optimized using a multi-task loss function to achieve model convergence.

[0021] Furthermore, the model inference module is also used to input the preoperative physiological asymmetry baseline feature set into the first temporal feature extraction branch of the personalized evaluation model based on Siamese networks, and output the preoperative baseline high-dimensional feature vector; input the postoperative motion feature set into the second temporal feature extraction branch of the model, and output the postoperative test high-dimensional feature vector; perform difference calculation on the two sets of high-dimensional feature vectors through feature difference layering, remove feature components that match the preoperative innate physiological asymmetry, obtain the diagnosis and treatment related feature difference vector, and calculate the corresponding quantization value through the regression output layer.

[0022] Furthermore, the result generation module is also used to calculate the rate of change of movement characteristics before and after treatment for each muscle region based on the quantitative values ​​of the symmetry deviation of the treatment-related muscle movements and the quantitative values ​​of the change in dynamic coordination. Based on the rate of change of movement characteristics, a heat map of the changes in facial muscle regions is generated to locate abnormal muscle regions whose rate of change exceeds a preset threshold. Based on the quantitative values ​​and the location results of abnormal muscle regions, a standardized evaluation report on the dynamic coordination changes of facial muscles before and after oral treatment is generated, and the evaluation results are output and stored locally.

[0023] Furthermore, the data acquisition module is also used to pre-set the standardized oral treatment-specific facial expression sequence. The sequence includes five sets of continuous action units: static neutral expression unit, gradient smiling action unit, continuous chewing action unit, continuous vowel pronunciation action unit, and mouth-opening-closing cycle action unit. A fixed action duration, action amplitude range, and time interval are set for each action unit, and the action requirements are displayed in real time through a visual guidance interface during video acquisition.

[0024] Additional features and advantages of this invention will be set forth in the description which follows, or may be learned by practicing the invention. Attached Figure Description

[0025] The technical solution and beneficial effects of the present invention will become apparent and readily understood from the following description in conjunction with the accompanying drawings, wherein:

[0026] Figure 1This is a flowchart of the overall workflow of the deep learning-based facial expression muscle dynamic collaborative evaluation system of the present invention, corresponding to the complete technical solution of independent claim 1 of the present invention;

[0027] Figure 2 This is a flowchart corresponding to Embodiment 1 of the present invention, specifically the standardized facial dynamic video acquisition process of the data acquisition module;

[0028] Figure 3 This is a flowchart corresponding to Embodiment 2 of the present invention, specifically the process of generating a three-dimensional facial coordinate dataset in the data preprocessing module;

[0029] Figure 4 This is a flowchart corresponding to Embodiment 3 of the present invention, specifically the process of extracting the three-dimensional displacement vector field of facial expression muscles in the vector field extraction module;

[0030] Figure 5 This is the flowchart corresponding to Embodiment 4 of the present invention, specifically the target muscle region division process of the feature extraction module based on anatomical priors;

[0031] Figure 6 This is the flowchart corresponding to Embodiment 5 of the present invention, specifically the process of generating bilateral muscle movement feature sets in the feature extraction module;

[0032] Figure 7 This is a flowchart of the process corresponding to Embodiment 6 of the present invention, specifically the construction and pre-training process of a personalized evaluation model based on Siamese networks;

[0033] Figure 8 This is the workflow diagram corresponding to Embodiment 7 of the present invention, specifically the feature difference and quantization value output process of the model inference module;

[0034] Figure 9 This is the workflow diagram corresponding to Embodiment 8 of the present invention, specifically the evaluation report generation and output process of the result generation module;

[0035] Figure 10 This is the workflow diagram corresponding to Embodiment 9 of the present invention, specifically the collection and execution process of standardized oral diagnosis and treatment-specific facial expression and action sequences. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0037] The deep learning-based facial expression muscle dynamic collaborative evaluation system disclosed in this invention will now be described with reference to the accompanying drawings.

[0038] likeFigure 1 As shown, a deep learning-based facial expression muscle dynamic collaborative evaluation system includes:

[0039] The data acquisition module is used to acquire multi-view high frame rate facial dynamic videos of subjects during the preoperative baseline period and multi-view high frame rate facial dynamic videos of the same subjects during the postoperative follow-up period. The video acquisition content during the preoperative baseline period and the postoperative follow-up period is the same standardized oral diagnosis and treatment-specific facial expression and movement sequence.

[0040] The data preprocessing module is used to perform three-dimensional spatial calibration on the multi-view high frame rate facial dynamic video during the preoperative baseline period and the multi-view high frame rate facial dynamic video during the postoperative follow-up period, and generate preoperative three-dimensional facial coordinate datasets and postoperative three-dimensional facial coordinate datasets respectively.

[0041] The vector field extraction module, based on the dense optical flow detection algorithm, performs temporal frame processing on the preoperative three-dimensional facial coordinate dataset and the postoperative three-dimensional facial coordinate dataset to extract the preoperative three-dimensional displacement vector field of facial muscles and the postoperative three-dimensional displacement vector field of facial muscles. The three-dimensional displacement vector field is the local motion vector field of facial muscles after eliminating rigid head movement.

[0042] The feature extraction module, based on the prior knowledge of oral and maxillofacial muscle anatomy, divides the target muscle regions into the three-dimensional displacement vector fields of the whole facial muscles before and after surgery, and extracts the physiological asymmetry benchmark feature sets of the corresponding muscle regions on both sides before surgery, and the motion feature sets of the corresponding muscle regions on both sides after surgery.

[0043] The model inference module is used to input the preoperative physiological asymmetry baseline feature set and the postoperative motion feature set into a pre-trained Siamese network-based personalized evaluation model. The Siamese network-based personalized evaluation model is equipped with a feature difference layer. Through the feature difference layer, the input preoperative baseline features and postoperative motion features are differentially calculated, and feature components that match the preoperative physiological asymmetry are removed. Only the muscle motion change components caused by the diagnosis and treatment operation are retained. Based on the retained feature components, the quantitative values ​​of the symmetry deviation of the diagnosis and treatment-related muscle motion and the quantitative values ​​of the dynamic coordination degree change are output. The dynamic coordination degree is a comprehensive quantitative value of the consistency of the movement phase, amplitude and trajectory of the same muscles on both sides in time.

[0044] The results generation module generates evaluation results of the dynamic synergistic changes of facial muscles before and after oral treatment for the corresponding subjects, based on the quantitative values ​​of the symmetry deviation of treatment-related muscle movements and the quantitative values ​​of dynamic synergy changes.

[0045] Example 1

[0046] like Figure 2As shown, the data acquisition module includes a multi-view acquisition unit. During the preoperative baseline period and postoperative follow-up period, the subjects are guided by a visual guidance interface to complete a standardized oral treatment-specific facial expression sequence according to a preset time sequence. The multi-view acquisition unit simultaneously acquires multi-angle facial dynamic videos.

[0047] Specifically:

[0048] In this embodiment, the multi-view acquisition unit consists of three industrial-grade global shutter CMOS cameras arranged horizontally in a straight line at equal intervals. The baseline distance between adjacent cameras is fixed at 120mm ± 5mm. The optical axis of the middle camera is perpendicular to the plane of the subject's face, and the angle between the optical axes of the left and right cameras and the optical axis of the middle camera is 15°. The imaging centers of the three cameras are at the same horizontal level. The core acquisition parameters of a single camera are fixed as follows: effective imaging resolution of 1920×1080, constant acquisition frame rate of 60fps in global shutter mode, pixel depth of 8bit, matching fixed-focus lens with a focal length of 16mm, fixed focusing distance of 1.0m, and fixed aperture value of F5.6.

[0049] To ensure the time synchronization accuracy of multi-view video, this embodiment adopts a hardware synchronization triggering scheme: A TTL level synchronization signal generator is set up, with the synchronization signal frequency consistent with the camera's frame rate. The synchronization signal is simultaneously connected to the external trigger input interfaces of three cameras to control the synchronized start of frame exposure actions of the three cameras; the single frame exposure time is fixed at 500μs, and the synchronization error of the multiple cameras is constrained by the following formula: ,in, The frame exposure trigger time for the intermediate camera. For the first The frame exposure trigger time of the side-mounted camera requires synchronization error of all cameras. If the threshold is exceeded, the system will automatically terminate the data collection and trigger a synchronization error alarm.

[0050] Standardized environment configuration must be completed before data collection:

[0051] The acquisition space was a dedicated, enclosed acquisition room free from light-sensitive interference. Two sets of symmetrically arranged ring-shaped soft LED light sources provided ambient lighting, with a fixed color temperature of 5500K and a color rendering index Ra≥95. The illuminance at the subject's face was uniformly controlled at 800lux±50lux, with no direct strong light or obvious facial shadows. A matte neutral gray background was used to avoid environmental reflections interfering with the imaging. The environmental configuration parameters were completely consistent between the preoperative baseline period and the postoperative follow-up period.

[0052] Before data collection, the subject's position must be standardized and calibrated.

[0053] An adjustable 3D headrest and chin rest with graduations are used to fix the subject's head, ensuring that the subject's facial plane is parallel to the imaging plane of the central camera, the subject's eyes are looking straight ahead, and the line connecting the upper edge of the tragus and the lower edge of the orbit (Frankfort plane) remains horizontal. The system constructs a standard head coordinate system based on real-time acquired rigid key facial feature points (bilateral outer canthi, bilateral inner canthi, and bilateral nasal root points), and calculates the deviation between the real-time head posture angle and the standard posture angle. The deviation calculation formula is as follows: In the formula, These are the head pitch angle, yaw angle, and roll angle, respectively. The subscript real represents the real-time measurement value, and std represents the preset standard value.

[0054] The visual guidance interface is deployed on a high-definition display screen directly in front of the subject, with a fixed straight-line distance of 1.2m between the display screen and the subject's face. The interface includes three functional areas: an action sequence guidance area, a real-time posture feedback area, and a data acquisition status prompt area. The action sequence guidance area displays a pre-set, standardized sequence of facial expressions specific to oral treatment, showing standard action templates, an action progress bar, and remaining time frame by frame, guiding the subject to complete the corresponding actions synchronously. The real-time posture feedback area displays real-time facial images of the subject captured by the central camera, overlaid with facial rigid feature point detection boxes and posture angle deviation values, providing real-time visual feedback on posture deviations. The data acquisition status prompt area displays the currently acquired action number, the number of frames acquired, the multi-camera synchronization status, and abnormal alarm information in real time.

[0055] A single complete data acquisition process is executed strictly according to a fixed time sequence:

[0056] Pre-verification phase:

[0057] The duration is fixed at 10 seconds. The interface guides the subject to keep their head fixed and a neutral and relaxed expression. The system simultaneously completes multi-camera synchronization verification, body position calibration verification, and illumination uniformity verification. Once all verification items pass, the system automatically enters the motion acquisition stage. If any verification fails, the system will continuously prompt for adjustment until the acquisition requirements are met.

[0058] Motion capture phase:

[0059] The motions were collected in strict accordance with the preset timing of the standardized oral treatment-specific facial expression sequence. After each motion unit was collected, a fixed 3-second relaxation interval was set to guide the subject to return to a neutral facial expression state and eliminate the influence of residual muscle tension from the previous motion on subsequent collections.

[0060] Verification phase complete:

[0061] After the full sequence of motion acquisition is completed, the system automatically performs frame synchronization integrity verification, image unobstructedness verification, and motion timing integrity verification. If the verification passes, the acquisition is completed, and the original video data acquired by the three cameras simultaneously is stored in lossless AVI format. The camera parameters, environmental parameters, body position parameters, and verification result metadata of this acquisition are recorded simultaneously. If the verification fails, the acquired data is automatically discarded, and the complete acquisition process is restarted.

[0062] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0063] Example 2

[0064] like Figure 3 As shown, the data preprocessing module performs single-target and stereo calibration on the multi-camera acquisition unit using a checkerboard calibration board, calculating the intrinsic parameters and distortion coefficients of each camera, as well as the extrinsic parameters between the multi-cameras. Based on the camera intrinsic parameters and distortion coefficients, distortion correction is performed on each frame of the acquired video to obtain a distortion-free frame sequence. Based on the extrinsic parameters of the multi-cameras, stereo correction and epipolar alignment are performed on the distortion-free frame sequence to generate a pixel-level aligned multi-view frame sequence. The multi-view frame sequence is then used for 3D reconstruction to generate the corresponding 3D facial coordinate dataset.

[0065] Specifically:

[0066] This embodiment uses a 12×9 black and white checkerboard calibration board. The side length of each square is 20mm±0.02mm, and the flatness tolerance is ≤0.01mm. The calibration process is completely consistent with the camera parameters and environmental configuration in the acquisition process.

[0067] First, complete the camera calibration:

[0068] Collect ≥20 images of calibration boards with different poses covering the entire field of view. Calculate the intrinsic parameter matrix and second-order radial + second-order tangential distortion coefficients of a single camera based on Zhang Zhengyou's calibration method. Calculate the rotation matrix R and translation vector T between multiple cameras through stereo calibration to complete the spatial pose calibration of the multi-camera system.

[0069] The camera intrinsic parameter matrix K is defined as follows: In the formula, The normalized focal length along the xy coordinate axes. These are the coordinates of the principal point of the image.

[0070] The distortion correction formula is:

[0071] In the formula, For the normalized coordinates of the distorted pixels, The radial distortion coefficient is... The tangential distortion coefficient is... These are the normalized coordinates after correction.

[0072] Based on the calibration parameters, distortion correction is performed frame by frame on the acquired video to generate a distortion-free frame sequence. Then, stereo correction and epipolar alignment are completed through multi-camera extrinsic parameters. After projection transformation, the multi-view images are mapped to the same plane to make the corresponding epipolar lines horizontally aligned and eliminate vertical parallax, generating a pixel-level aligned multi-view frame sequence.

[0073] For the aligned frame sequence, a semi-global stereo matching algorithm is used to calculate the pixel-level disparity map, and the three-dimensional spatial coordinates of each facial pixel are calculated using the triangulation principle. The core calculation formula is as follows: In the formula, The pixel depth value. For multi-camera baseline distance, The values ​​represent pixel disparity; ultimately, a temporally continuous preoperative and postoperative 3D facial coordinate dataset is generated.

[0074] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0075] Example 3

[0076] like Figure 4 As shown, the dense optical flow detection algorithm of the vector field extraction module performs pixel matching on continuous temporal frames of the preoperative and postoperative 3D facial coordinate datasets. Using pre-extracted key facial feature points as rigid constraints, it calculates the 3D displacement vector of each facial pixel in each frame. All 3D displacement vectors are smoothed by temporal moving average, and then Gaussian filtering is used to remove high-frequency noise and eliminate displacement interference caused by non-muscle movements, generating a temporally continuous 3D displacement vector field.

[0077] Specifically:

[0078] This embodiment uses the Farneback dense optical flow algorithm to first extract eight rigid facial feature points (bilateral outer corners of the eyes, bilateral inner corners of the eyes, bilateral tragus points, and nasal root points) from the temporal frames of the three-dimensional facial coordinate dataset, as constraint references to eliminate global rigid head motion.

[0079] By matching rigid feature points between frames, the global rotation matrix and translation vector of adjacent frames are calculated to complete global motion compensation, eliminate non-muscle motion displacement caused by overall head translation and rotation, and obtain a temporal frame sequence containing only local facial muscle motion.

[0080] Dense optical flow calculations are performed on adjacent temporal frames to solve for the two-dimensional optical flow vector of each facial pixel. This is then combined with three-dimensional coordinate mapping to obtain the three-dimensional displacement vector. The core calculation formula is: In the formula, For each pixel, there is a two-dimensional optical flow vector. The coefficient matrix is ​​the polynomial expansion matrix of the pixel neighborhood. The first The frame corresponds to the polynomial coefficient vector of the neighborhood; combined with the pixel's three-dimensional coordinates, the two-dimensional optical flow is mapped into a three-dimensional displacement vector. .

[0081] For the full-face 3D displacement vector, first perform temporal dimension moving average smoothing, with a fixed window size of 5 frames and a step size of 1 frame;

[0082] Then, a 5×5 Gaussian kernel is used for spatial domain filtering with a standard deviation σ=1.2 to filter out high-frequency noise and non-muscle movement jitter, and finally generate a temporally continuous three-dimensional displacement vector field of facial expression muscles before and after surgery.

[0083] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0084] Example 4

[0085] like Figure 5 As shown, the feature extraction module is used to pre-establish facial muscle region mapping templates related to orthodontic and restorative treatment based on prior knowledge of oral and maxillofacial muscle anatomy. The mapping templates include five bilateral muscle regions with the same name: orbicularis oris, zygomaticus major, zygomaticus minor, levator labii superioris, and masseter, as well as the facial pixel coordinate mapping relationship corresponding to each muscle region. Based on the facial muscle region mapping templates, the target muscle regions are divided, a one-to-one correspondence between bilateral muscle regions with the same name is established, and the three-dimensional displacement vector data corresponding to each divided region is extracted.

[0086] Specifically:

[0087] This embodiment is based on the internationally accepted 68-point facial key feature point system, combined with prior knowledge of oral and maxillofacial muscle anatomy, and pre-establishes a facial muscle region mapping template. It clarifies the anchoring feature points, boundary ranges and pixel coordinate mapping rules of 5 sets of bilateral identical muscle regions, and establishes a one-to-one correspondence between bilateral muscle regions with the midsagittal plane of the face as the axis of symmetry.

[0088] The core definitions of the 5 muscle groups are as follows:

[0089] Orbicularis oris muscle:

[0090] Anchor the perioral feature points 48-67. The area is a closed region extending 15 pixels outward from the anchor point, covering the entire muscle range of the upper and lower lips.

[0091] Zygomaticus major muscle:

[0092] The left and right sides are anchored with feature points 0, 1, 2, 31, 36, 48 and 15, 16, 17, 35, 45, 54 respectively. The area is a convex polygonal closed region formed by the anchor points.

[0093] Zygomaticus minor:

[0094] Feature points 36, 37, 39, 48 and 42, 44, 45, 54 are anchored on the left and right sides respectively. The area is a convex polygonal closed region formed by the anchor points.

[0095] Lip muscles:

[0096] Feature points 31, 32, 39, 48 and 34, 35, 42, 54 are anchored on the left and right sides respectively. The region is a convex polygonal closed area formed by the anchor points.

[0097] Masseter muscle:

[0098] Feature points 0, 4, 8, 10 and 15, 11, 12 and 16 are anchored on the left and right sides respectively. The region is a closed convex polygon formed by the anchor points.

[0099] The formula for symmetric mapping of pixel coordinates on both sides is: In the formula, The width of the facial region image. The pixel coordinates of the left region are... The corresponding pixel coordinates on the right side are used to achieve a pixel-level one-to-one correspondence between the muscle regions on both sides.

[0100] Binary masks for each muscle region are generated based on the mapping template. The three-dimensional displacement vectors corresponding to all pixels in the mask are extracted to form displacement vector datasets for the left and right muscle regions with the same name, thus completing the division of the target muscle region and the extraction of corresponding data.

[0101] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0102] Example 5

[0103] like Figure 6 As shown, the feature extraction module is used to perform temporal alignment of the three-dimensional displacement vector data of the bilateral corresponding muscle regions in the three-dimensional displacement vector field of the preoperative full-face facial expression muscles, and to match the key action nodes of the standardized oral treatment-specific facial expression action sequence; calculate the statistical mean and distribution characteristics of the difference in motion amplitude, motion phase, and motion trajectory deviation of the bilateral corresponding muscle regions under each key action node, combine the statistical results corresponding to all key action nodes to generate the preoperative physiological asymmetry benchmark feature set, and use the same logic to generate the postoperative motion feature set.

[0104] Specifically:

[0105] This embodiment uses the preset start point, peak point, and end point of the standardized oral treatment-specific facial expression action sequence as key action nodes. The Dynamic Time Warping (DTW) algorithm is used to perform frame-level alignment of the three-dimensional displacement vector temporal data of the corresponding muscle regions on both sides, matching the temporal position of the key action nodes and eliminating the temporal deviation caused by the difference in action execution speed.

[0106] After alignment, calculate the core motion deviation characteristics of the bilateral corresponding muscle regions at each key motion node using the following formula:

[0107] The formula for calculating the range of motion difference of a single muscle region is as follows: In the formula, This represents the total number of pixels within the muscle region. For the first in the region A 3D displacement vector of pixels. The vector magnitude;

[0108] The difference in amplitude between the two sides is: In the formula, The range of motion for the corresponding muscle regions on the left and right sides are shown respectively.

[0109] Motion phase difference In the formula, These are the frame numbers corresponding to the peak displacement values ​​of the left and right muscle regions, respectively. This represents the total number of frames for the current action unit.

[0110] Motion trajectory deviation:

[0111] In the formula, These are the 3D displacement trajectory coordinates of the corresponding pixels on the left and right sides, respectively.

[0112] For all key action nodes The statistical mean, standard deviation, and range are calculated separately to form the distribution characteristics. The statistical results of all key nodes are combined according to fixed dimensions to generate the preoperative physiological asymmetry benchmark feature set, and the postoperative motion feature set is generated using completely consistent calculation logic.

[0113] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0114] Example 6

[0115] like Figure 7As shown, the model inference module is used to pre-build a personalized evaluation model based on Siamese networks. The model includes two temporal feature extraction branches with completely shared weights, a feature difference layer, and a regression output layer. The temporal feature extraction branch adopts a 3-layer LSTM structure to extract the temporal dependency features of the input features. The feature difference layer is used to calculate the difference between the high-dimensional feature vectors output by the two branches. The model is pre-trained using paired pre- and post-operative facial expression muscle movement annotation datasets, and the model parameters are optimized using a multi-task loss function to achieve model convergence.

[0116] Specifically:

[0117] In this embodiment, a personalized evaluation model based on Siamese networks is pre-constructed. The model consists of two time-series feature extraction branches with completely shared weights, feature difference layering, and regression output layer connected in series. The input dimension matches the dimensions of the preoperative baseline feature set and the postoperative motion feature set, and is fixed at 15 dimensions (5 groups of muscles × 3 deviation features).

[0118] The temporal feature extraction branch adopts a 3-layer stacked LSTM structure, with a fixed number of 128 hidden units in each layer. The activation function is Tanh, and the recurrent activation function is Sigmoid. A Dropout coefficient of 0.2 is set in each layer to prevent overfitting. Finally, a 64-dimensional high-dimensional feature vector is output to represent the temporal dependence features of muscle movement.

[0119] The feature difference layer performs element-wise difference calculation, as shown in the formula:

[0120] In the formula, This is a high-dimensional feature vector extracted from the preoperative baseline features through branching. This is a high-dimensional feature vector extracted from postoperative motion features via branching. This is the difference vector of diagnosis-related features after removing physiological asymmetries.

[0121] The regression output layer adopts a 2-layer fully connected structure, with a hidden layer dimension of 32 and an output layer dimension of 2, corresponding to the quantified values ​​of muscle movement symmetry deviation and dynamic coordination change, respectively. The output layer activation function adopts Sigmoid, and the output value range is fixed in [0,1].

[0122] The model pre-training used a paired facial expression muscle movement dataset from the preoperative baseline and postoperative follow-up periods of the same subject, with a sample size of ≥1000 groups. All samples came from clinical subjects undergoing orthodontic or prosthodontic treatment, and the collection procedures, action sequences, and environmental parameters were completely identical before and after the procedure. The inclusion criteria were: completion of complete preoperative baseline and 6-month postoperative follow-up data collection, no history of facial nerve disease, no history of maxillofacial trauma, and no other interventions affecting facial muscle movement before or after treatment. The dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. This division process ensured that paired preoperative and postoperative samples from the same subject belonged to the same subset of the dataset, avoiding data leakage.

[0123] The supervisory label is the gold standard quantitative value of the symmetry deviation and dynamic coordination change of treatment-related muscle movements, jointly annotated by two or more associate chief physicians or above in the Department of Oral and Maxillofacial Surgery. The annotation process adopts a double-blind independent annotation method. Two physicians independently quantify and score the same set of preoperative and postoperative paired samples, with the scoring range being [0,1], which perfectly matches the model output dimension. After the annotation is completed, the intragroup correlation coefficient (ICC) consistency is checked, requiring ICC ≥ 0.85. If it is lower than the threshold, a third chief physician reviews and decides, and finally determines the gold standard quantitative value to ensure the accuracy and consistency of the annotation results.

[0124] Training employs a multi-task loss function, composed of mean squared error loss, cosine similarity loss, and L2 regularization term, as shown in the formula:

[0125] In the formula, The mean squared error loss is the deviation between the fitted output value and the gold standard. Cosine similarity loss is used to constrain the consistency of feature extraction in the two branches; The L2 regularization term is used to prevent overfitting; the weighting coefficients are fixed at α=0.7, β=0.2, and γ=0.1.

[0126] Training was performed using the Adam optimizer with an initial learning rate of 1e. -4 The batch size is 32, the maximum number of iterations is 200, and an early stopping strategy is set: if the validation set loss does not decrease for 10 consecutive iterations, the training is terminated; the model convergence criterion is that the validation set mean absolute error (MAE) is ≤0.03. After the condition is met, the model weights are saved and the pre-training is completed.

[0127] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0128] Example 7

[0129] like Figure 8As shown, the model inference module is also used to input the preoperative physiological asymmetry baseline feature set into the first temporal feature extraction branch of the Siamese network-based personalized evaluation model, and output the preoperative baseline high-dimensional feature vector; input the postoperative motion feature set into the second temporal feature extraction branch of the model, and output the postoperative test high-dimensional feature vector; perform difference calculation on the two sets of high-dimensional feature vectors through feature difference layering, remove feature components that match the preoperative innate physiological asymmetry, obtain the diagnosis and treatment related feature difference vector, and calculate the corresponding quantization value through the regression output layer.

[0130] Specifically:

[0131] Before inference, the input preoperative physiological asymmetry baseline feature set and postoperative motion feature set are subjected to min-max normalization to map all feature values ​​to the [0,1] interval.

[0132] Load the pre-trained Siamese network model, input the normalized preoperative baseline feature set into the first temporal feature extraction branch, and extract temporal dependent features layer by layer through a 3-layer LSTM with shared weights, outputting a 64-dimensional preoperative baseline high-dimensional feature vector. The normalized postoperative motion feature set is input into the second temporal feature extraction branch with completely equal weights, and a 64-dimensional postoperative high-dimensional feature vector to be tested is output simultaneously. .

[0133] Feature difference layering first calculates the inherent matching components of the two sets of feature vectors using cosine similarity, then performs element-wise difference calculation to remove feature components that match preoperative innate physiological asymmetry, resulting in feature difference vectors that are only relevant to the diagnostic and treatment procedures. The core formula is: In the formula, The cosine similarity between the two sets of feature vectors represents the proportion of preoperative inherent asymmetry in postoperative features.

[0134] The feature difference vector The input regression output layer completes the dimension mapping through a 2-layer fully connected network, and finally outputs 2 standardized quantitative values: the quantitative value of the symmetry deviation of the treatment-related muscle movement S and the quantitative value of the change in dynamic coordination C. The output value range is [0,1]. The smaller the value of S, the better the muscle symmetry after treatment. The closer the value of C is to 1, the more significant the improvement in the dynamic coordination of the muscles after treatment.

[0135] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0136] Example 8

[0137] like Figure 9As shown, the results generation module is also used to calculate the rate of change of movement characteristics before and after treatment for each muscle region based on the quantitative values ​​of the symmetry deviation of muscle movement and the quantitative values ​​of the change in dynamic coordination. Based on the rate of change of movement characteristics, a heat map of the changes in facial muscle regions is generated to locate abnormal muscle regions whose rate of change exceeds a preset threshold. Based on the quantitative values ​​and the location results of abnormal muscle regions, a standardized evaluation report on the dynamic coordination changes of facial muscles before and after oral treatment is generated, and the evaluation results are output and stored locally.

[0138] Specifically:

[0139] Based on the output quantitative values ​​of symmetry deviation and dynamic synergy change of the diagnostic and treatment-related muscle movements, combined with the movement characteristics of 5 bilateral muscle regions, the rate of change of movement characteristics of a single muscle region before and after diagnosis and treatment is calculated using the following formula: In the formula, For the first Rate of change of motor characteristics in muscle groups These represent the mean difference in bilateral range of motion of the muscles before and after surgery in this group. It corresponds to 5 groups of target facial muscles.

[0140] Based on each muscle region The values ​​are mapped to RGB color values ​​using the Jet pseudo-color mapping rule, generating a heatmap of facial muscle region changes. The mapping range is fixed at [-100%, +100%], where negative values ​​represent improved muscle movement symmetry and positive values ​​represent worsened symmetry.

[0141] The preset threshold for abnormal change rate is ±30%, when a single muscle area When an abnormal muscle region is identified, the system automatically marks the region's location, rate of change value, and corresponding muscle name, thus completing the location of the abnormal region.

[0142] Based on core quantitative values, abnormal area location results, and change heatmaps, a standardized evaluation report is generated. The report includes subject information, preoperative baseline data, postoperative quantitative results, comparison of changes before and after treatment, abnormal area analysis, and comprehensive evaluation conclusions. The report is output in PDF format, and the original quantitative data, feature sets, and heatmaps are simultaneously encrypted and stored locally in JSON format. The data is bound to the subject's unique ID to achieve full-process traceability.

[0143] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0144] Example 9

[0145] like Figure 10As shown, the data acquisition module is also used to pre-set standardized oral treatment-specific facial expression sequences. The sequence includes five sets of continuous action units: static neutral expression unit, gradient smiling action unit, continuous chewing action unit, continuous vowel pronunciation action unit, and mouth-closing-opening-closing cycle action unit. Fixed action duration, action amplitude range, and time interval are set for each action unit, and the action requirements are displayed in real time through a visual guidance interface during video acquisition.

[0146] Specifically:

[0147] In this embodiment, a standardized facial expression sequence for oral treatment is pre-set, with a fixed total duration of 90 seconds. Five sets of action units are executed sequentially according to a fixed time sequence, with a fixed 3-second relaxation interval between adjacent units to guide the subject to restore a neutral expression and eliminate residual muscle tension. Throughout the data collection process, the standard action template, progress bar, and beat prompts are displayed synchronously through a visual guidance interface to ensure the consistency of action execution.

[0148] The specific execution standards for the 5 action units are as follows:

[0149] Static neutral expression unit:

[0150] The duration is fixed at 10 seconds. Subjects are required to look straight ahead, close their lips naturally, lightly touch their upper and lower teeth, and maintain a completely relaxed state with no active facial muscle contraction. The system constrains the displacement of key facial feature points to be ≤2 pixels per frame in real time. If the displacement exceeds this limit, the system prompts for re-execution to ensure the stability of baseline data.

[0151] Gradient-smile action unit:

[0152] The duration is fixed at 15 seconds, divided into 3 gradient stages, with each stage lasting 5 seconds. The movements are performed sequentially: a slight upturn of the corners of the mouth at 10%, a natural smile at 50%, and a maximum smile showing teeth at 100%. Each stage involves a uniform speed for initiation, holding, and resetting of the movement. Based on a neutral expression, the displacement range of the corners of the mouth is quantified as 2-4 mm, 8-12 mm, and 18-22 mm, respectively.

[0153] Continuous chewing action unit:

[0154] The duration is fixed at 20 seconds, simulating the chewing action of the posterior teeth on one side. The chewing frequency is fixed at 1 time per second, for a total of 20 cycles. Each cycle consists of opening the mouth, biting, and repositioning. The mouth opening amplitude between the incisors is fixed at 10±2mm. 10 consecutive chewing actions are performed on each side without any pauses.

[0155] Continuous vowel articulation action unit:

[0156] With a fixed duration of 20 seconds, the system sequentially pronounces the five standard vowels: / a / , / o / , / e / , / i / , and / u / , with each monophthong sustained for 4 seconds. The system constrains the displacement fluctuation of the characteristic points in the orbicularis oris muscle region during pronunciation to ≤3 pixels, ensuring stable mouth shape without deformation.

[0157] Closed-opening cyclic action unit:

[0158] The duration is fixed at 15 seconds, and the mouth is opened and closed at a constant speed. The cycle frequency is fixed at 1 time every 3 seconds, for a total of 5 cycles. The maximum mouth opening amplitude between the incisors in a single cycle is fixed at 30±2mm.

[0159] For the remaining technical details of this embodiment, please refer to the above technical details, which will not be repeated in this embodiment.

[0160] In summary, the deep learning-based facial expression muscle dynamic collaborative evaluation system disclosed in this invention can eliminate the interference of subjects' innate physiological facial asymmetry, accurately quantify the changes in facial expression muscle motor function caused by oral diagnosis and treatment, improve the accuracy of efficacy evaluation, and provide reliable digital basis for optimizing clinical diagnosis and treatment plans.

[0161] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A deep learning-based dynamic collaborative evaluation system for facial expression muscles, characterized in that, include: The data acquisition module is used to acquire multi-view high frame rate facial dynamic videos of the subjects during the preoperative baseline period and multi-view high frame rate facial dynamic videos of the same subjects during the postoperative follow-up period. The video acquisition content during the preoperative baseline period and the postoperative follow-up period is the same standardized oral diagnosis and treatment-specific facial expression and movement sequence. The data preprocessing module is used to perform three-dimensional spatial calibration on the multi-view high frame rate facial dynamic video during the preoperative baseline period and the multi-view high frame rate facial dynamic video during the postoperative follow-up period, and generate a preoperative three-dimensional facial coordinate dataset and a postoperative three-dimensional facial coordinate dataset, respectively. The vector field extraction module, based on the dense optical flow detection algorithm, performs temporal frame processing on the preoperative three-dimensional facial coordinate dataset and the postoperative three-dimensional facial coordinate dataset to extract the preoperative three-dimensional displacement vector field of the facial muscles and the postoperative three-dimensional displacement vector field of the facial muscles. The feature extraction module, based on prior knowledge of oral and maxillofacial muscle anatomy, divides the target muscle regions of the three-dimensional displacement vector field of the preoperative and postoperative three-dimensional displacement vector fields of the facial muscles, and extracts the physiological asymmetry benchmark feature set of the corresponding muscle regions on both sides before surgery, and the motion feature set of the corresponding muscle regions on both sides after surgery. The model inference module inputs the preoperative physiological asymmetry baseline feature set and the postoperative motor feature set into the pre-trained twin network-based personalized evaluation model, and outputs the quantitative values ​​of the symmetry deviation of diagnosis-related muscle movements and the quantitative values ​​of dynamic coordination changes after removing the innate physiological asymmetry. The results generation module generates evaluation results of the dynamic synergistic changes of facial muscles before and after oral treatment for the corresponding subjects, based on the quantitative values ​​of the symmetry deviation of the treatment-related muscle movements and the quantitative values ​​of the dynamic synergistic changes.

2. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 1, characterized in that, The data acquisition module includes a multi-view acquisition unit. During the preoperative baseline period and postoperative follow-up period, the subject is guided by a visual guidance interface to complete the standardized oral treatment-specific facial expression and movement sequence according to a preset time sequence. The multi-view acquisition unit simultaneously acquires multi-angle facial dynamic videos.

3. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 2, characterized in that, The data preprocessing module performs single-target and stereo calibration on the multi-camera acquisition unit using a checkerboard calibration plate, and calculates the intrinsic parameters, distortion coefficients of each camera, and extrinsic parameters between the multi-cameras. Based on the distortion coefficients of the camera, distortion correction is performed on each frame of the acquired video to obtain a distortion-free frame sequence. Based on the extrinsic parameters of the multi-view camera, stereo correction and epipolar alignment are performed on the distortion-free frame sequence to generate a pixel-level aligned multi-view frame sequence. The multi-view frame sequence is then reconstructed in three dimensions to generate a corresponding three-dimensional facial coordinate dataset.

4. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 3, characterized in that, The dense optical flow detection algorithm of the vector field extraction module performs pixel matching on the continuous temporal frames of the preoperative three-dimensional facial coordinate dataset and the postoperative three-dimensional facial coordinate dataset, and calculates the three-dimensional displacement vector of each facial pixel in each frame with the pre-extracted key facial feature points as rigid constraints. All three-dimensional displacement vectors are smoothed by moving average in the time dimension, and then Gaussian filtering is used to remove high-frequency noise and eliminate displacement interference caused by non-muscle movement, generating a time-continuous three-dimensional displacement vector field.

5. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 4, characterized in that, The feature extraction module is used to pre-establish facial muscle region mapping templates related to orthodontic and prosthodontic treatment based on prior knowledge of oral and maxillofacial muscle anatomy. The mapping templates include five bilateral muscle regions with the same name: orbicularis oris, zygomaticus major, zygomaticus minor, levator labii superioris, and masseter, as well as the facial pixel coordinate mapping relationship corresponding to each muscle region. Based on the facial muscle region mapping template, the target muscle region is divided, a one-to-one correspondence between bilateral muscle regions with the same name is established, and the three-dimensional displacement vector data corresponding to each divided region is extracted.

6. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 5, characterized in that, The feature extraction module is used to perform temporal alignment of the three-dimensional displacement vector data of the bilateral corresponding muscle regions in the preoperative full-face facial expression muscle three-dimensional displacement vector field, and to match the key action nodes of the standardized oral treatment-specific facial expression action sequence; calculate the statistical mean and distribution characteristics of the difference in motion amplitude, the difference in motion phase, and the deviation in motion trajectory of the bilateral corresponding muscle regions under each key action node, combine the statistical results corresponding to all key action nodes to generate the preoperative physiological asymmetry benchmark feature set, and use the same logic to generate the postoperative motion feature set.

7. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 6, characterized in that, The model inference module is used to pre-build the personalized evaluation model based on Siamese network. The model includes two time-series feature extraction branches with completely shared weights, a feature difference layer, and a regression output layer. The time-series feature extraction branch adopts a 3-layer LSTM structure to extract the time-series dependency features of the input features. The feature difference layer is used to calculate the difference between the high-dimensional feature vectors of the two branch outputs; the model is pre-trained using paired pre- and post-operative facial expression muscle motion annotation datasets, and the model parameters are optimized using a multi-task loss function to achieve model convergence.

8. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 7, characterized in that, The model inference module is also used to input the preoperative physiological asymmetry baseline feature set into the first temporal feature extraction branch of the personalized evaluation model based on Siamese network, and output the preoperative baseline high-dimensional feature vector; input the postoperative motion feature set into the second temporal feature extraction branch of the model, and output the postoperative test high-dimensional feature vector; By performing difference calculations on two sets of high-dimensional feature vectors through feature difference layering, feature components that do not match preoperative innate physiological asymmetry are removed to obtain the feature difference vector related to diagnosis and treatment. The corresponding quantized value is then obtained by calculating through the regression output layer.

9. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 8, characterized in that, The result generation module is also used to calculate the rate of change of motion characteristics before and after treatment for each muscle region based on the quantitative values ​​of the symmetry deviation of the diagnosis-related muscle movement and the quantitative values ​​of the change in dynamic coordination. Based on the motion feature change rate data, a change heat map of the facial muscle region is generated to locate abnormal muscle regions whose change rate exceeds a preset threshold. Based on the quantified values ​​and the results of abnormal muscle region localization, a standardized evaluation report on the dynamic synergistic changes of facial muscles before and after oral treatment is generated, and the evaluation results are output and stored locally.

10. The deep learning-based facial expression muscle dynamic collaborative evaluation system as described in claim 9, characterized in that, The data acquisition module is also used to pre-set the standardized oral treatment-specific facial expression sequence. The sequence includes five sets of continuous action units: static neutral expression unit, gradient smiling action unit, continuous chewing action unit, continuous vowel pronunciation action unit, and mouth-closing and mouth-opening cycle action unit. A fixed action duration, action amplitude range, and time interval are set for each action unit, and the action requirements are displayed in real time through a visual guidance interface during the video acquisition process.

Images