A video face slimming method and system based on key point partition control

Abstract

The application discloses a video face slimming method and system based on key point partition control, and belongs to the field of computer vision and digital image processing. The method comprises the following steps: acquiring and analyzing video attribute information, initializing a video processing flow; reading image frames frame by frame and performing face key point detection; dividing a face into two sub-areas based on a data set; adaptively adjusting face slimming strength according to width difference; applying a local deformation algorithm using the adjusted parameters; and synchronously processing the processed image frames and original audio stream by time stamp to generate a final video output. The application adopts face key point detection and partition control technology, realizes fine face slimming of the face area, effectively solves the problem of audio-visual asynchronization in the video by using time stamp synchronous processing, can provide natural and personalized face beautification effect while ensuring real-time performance, and significantly improves user experience and video processing quality.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing, and in particular to a video face slimming method and system based on key point partitioning control. Background Technology

[0002] Digital image processing technology, especially in mobile internet and social media applications, has become a core function. Among these, beautification technologies for portraits, particularly faces, have developed rapidly, encompassing various functions such as whitening, skin smoothing, eye enlargement, and face slimming. Video face slimming, a popular beautification feature, aims to modify the contours of faces in video frames in real time through algorithms, making them appear slimmer. This typically relies on the accurate recognition of facial features using computer vision technology and the geometric transformations of images using computer graphics technology.

[0003] Existing video face slimming technologies typically employ a solution where face detection algorithms locate the face within an image frame and extract key feature points of the facial contour. Based on these feature points, a global or local image deformation algorithm, such as mesh deformation or liquefaction filtering, is applied to shrink the contours on both sides of the cheeks inward to a certain extent. In most implementations, the user can set a face slimming intensity parameter, which will be applied uniformly to the entire facial contour or symmetrically to the left and right cheeks.

[0004] However, existing technologies have some inherent technical flaws in practical applications. Since most people's faces are not perfectly symmetrical, applying a uniform or symmetrical slimming intensity often ignores this inherent asymmetry, and may even accentuate it after slimming, resulting in an unnatural effect. Furthermore, in consecutive video frames, minute errors in keypoint detection accumulate frame by frame, causing discontinuous jumps or flickering in the deformation effect. Improperly handled boundaries between deformed and unprocessed areas can also create obvious and unnatural visual demarcation lines. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a video face slimming method and system based on key point partitioning control. It employs a technical solution involving facial key point detection, left and right facial partitioning, width difference calculation, and adaptive adjustment of face slimming intensity. This enables personalized and differentiated face slimming processing for asymmetrical facial shapes, effectively enhancing the naturalness and harmony of video beautification effects.

[0006] The above objectives can be achieved through the following approach:

[0007] A video face-slimming method based on keypoint partitioning control includes: acquiring a video file and parsing video attribute information; initializing the video processing flow and generating initialization processing parameters, wherein the initialization processing parameters include video resolution, frame rate, and encoding format; reading image frames of the video file frame by frame, and performing facial keypoint detection on each frame according to the frame rate set in the initialization processing parameters to generate a facial keypoint dataset; dividing the facial region into left and right processing sub-regions based on the facial keypoint dataset to generate partitioning configuration data; calculating the width difference between the left and right processing sub-regions according to the partitioning configuration data, and adaptively adjusting the face-slimming intensity to generate adjusted face-slimming parameters; applying a local deformation algorithm to the left and right processing sub-regions respectively using the adjusted face-slimming parameters to generate face-slimmed image frames; performing time-stamp synchronization processing between the face-slimmed image frames and the original audio stream in the video file, and generating the final video output based on the encoding format in the initialization processing parameters.

[0008] Optionally, the steps of acquiring the video file and parsing the video attribute information, initializing the video processing flow, and generating initialization processing parameters include: checking the video file to verify its existence and readability; reading the metadata stream inside the video file and initializing the video processing flow to obtain initialization processing parameters including video resolution, frame rate, total number of frames, and encoding format.

[0009] Optionally, the step of reading the image frames of the video file frame by frame and performing facial landmark detection on each frame according to the frame rate set in the initialization processing parameters includes: using the face detection function to identify the face position in the image frame and generating face region data; extracting facial feature points within the range defined by the face region data to generate a preliminary key point set; and performing cross-frame temporal smoothing filtering on the preliminary key point set to suppress inter-frame jitter and generate a facial landmark dataset.

[0010] Optionally, dividing the facial region into two processing sub-regions (left and right) includes: extracting jaw contour points from the facial key point dataset to generate cheek contour data; determining the facial midline based on the nose tip point and chin center point in the facial key point dataset to generate midline data; and using the midline data to segment the cheek contour data to generate partition configuration data containing the left cheek region and the right cheek region.

[0011] Optionally, the step of calculating the width difference between the left and right processing sub-regions and adaptively adjusting the face-slimming intensity includes: calculating a first width value of the left cheek region and a second width value of the right cheek region based on the partition configuration data to generate width difference data; comparing the width difference data with a preset symmetry threshold to determine the asymmetry of the face shape and generating an asymmetry marker; and generating differentiated adjusted face-slimming parameters based on the asymmetry marker and the user-set basic face-slimming intensity.

[0012] Optionally, based on the asymmetry indicator and the user-defined basic face-slimming intensity, the process includes: if the asymmetry indicator indicates a face shape asymmetry, calculating the left and right adjustment factors based on the width difference data to generate adjustment factor data; and combining the basic face-slimming intensity with the adjustment factor data to generate adjusted face-slimming parameters.

[0013] Optionally, applying the local deformation algorithm to the left and right processing sub-regions respectively includes: for each processing sub-region, calculating the distance between the pixel point in the sub-region and the geometric center of the sub-region, and generating distance weight data; combining the distance weight data with the adjusted face-slimming parameters, calculating the displacement vector for each pixel point, and generating displacement mapping data; and based on the displacement mapping data, resampling the pixels through inverse mapping and bilinear interpolation algorithms to generate a face-slimming image frame.

[0014] Optionally, the step of combining the distance weight data and the adjusted face-slimming parameters to calculate the displacement vector for each pixel includes: performing boundary checks on the calculated displacement vectors to prevent pixel coordinates from exceeding the image range and generating effective displacement data; performing Gaussian blur processing on the displacement vectors in the effective displacement data that are located at the edge of the processing sub-region to smooth the boundary between the deformed region and the unprocessed region and generating displacement mapping data.

[0015] Optionally, the process of synchronizing the face-slimming image frames with the original audio stream in the video file using timestamps, and generating the final video output based on the encoding format in the initialization processing parameters, includes: extracting the playback time of each audio packet from the original audio stream in the video file to generate audio timestamp data; matching the timestamp in the audio timestamp data with each face-slimming image frame to generate adjusted timestamp data; maintaining a mapping table that records the correspondence between the face-slimming image frames and the adjusted timestamp data, and using the mapping table during video encapsulation to generate the final video output based on the encoding format in the initialization processing parameters.

[0016] Based on the same inventive concept, this invention also provides a video face-slimming system based on key point partitioning control. The system includes: an initialization module for acquiring video files and parsing video attribute information, initializing the video processing flow, and generating initialization processing parameters, wherein the initialization processing parameters include video resolution, frame rate, and encoding format; a key point detection module for reading image frames frame by frame and performing facial key point detection according to the frame rate set in the initialization processing parameters, generating a facial key point dataset; a partitioning module for dividing the facial region into left and right processing sub-regions based on the facial key point dataset, generating partitioning configuration data; a parameter adjustment module for calculating the width difference and adaptively adjusting the face-slimming intensity according to the partitioning configuration data, generating adjusted face-slimming parameters; a deformation processing module for applying a local deformation algorithm to the left and right processing sub-regions respectively using the adjusted face-slimming parameters, generating face-slimmed image frames; and a synchronization output module for performing time-stamp synchronization processing between the face-slimmed image frames and the original audio stream, and generating the final video output based on the encoding format in the initialization processing parameters.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. This invention achieves highly personalized and natural-looking video face slimming through a strategy of dynamic partitioning based on key points and adaptive parameter adjustment. It can accurately identify and quantify the width difference between the left and right sides of the face, and then apply stronger deformation to the wider side and weaker deformation to the narrower side. This not only achieves face slimming but also visually corrects asymmetry, making the processed facial contour more harmonious and balanced, avoiding the stiffness caused by the one-size-fits-all approach of traditional methods.

[0019] 2. This invention significantly improves the stability and visual smoothness of video face slimming processing. By performing cross-frame temporal smoothing filtering on the initially detected facial key points, it effectively suppresses key point jitter caused by subtle changes in pose and lighting in the video sequence, ensuring the continuity of the deformation reference. Simultaneously, by blurring the displacement vectors at the edges of the deformed region, a seamless transition between the processed and unprocessed areas is achieved, eliminating abrupt visual boundaries and resulting in a smooth and consistent final effect.

[0020] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a video face-slimming method based on key point partitioning control according to an embodiment of the present invention.

[0023] Figure 2 This is a comparison diagram of the average inter-frame displacement of key points in an embodiment of the present invention.

[0024] Figure 3 This is a comparison diagram of the asymmetry correction effect under different head postures in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of a video face-slimming system based on key point partitioning control according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Reference Figure 1 One embodiment of this invention proposes a video face slimming method based on key point partitioning control, which can perform personalized and differentiated face slimming processing on asymmetrical faces in videos. This invention employs facial key point detection and partitioning control technology to achieve refined face slimming of facial regions, and effectively solves the problem of audio-visual asynchrony in videos by utilizing timestamp synchronization processing. It can provide natural and personalized face beautification effects while ensuring real-time performance, significantly improving user experience and video processing quality.

[0028] The method described in this embodiment specifically includes:

[0029] The video file is acquired and its attribute information is parsed. The video processing flow is initialized and initialization processing parameters are generated, including video resolution, frame rate and encoding format.

[0030] The video file is read frame by frame, and facial landmark detection is performed on each frame according to the frame rate set in the initialization processing parameters to generate a facial landmark dataset.

[0031] Based on the aforementioned facial key point dataset, the facial region is divided into two processing sub-regions, left and right, to generate partition configuration data;

[0032] Based on the partition configuration data, the width difference between the left and right processing sub-regions is calculated, and the face-slimming intensity is adaptively adjusted to generate the adjusted face-slimming parameters;

[0033] Using the adjusted face-slimming parameters, a local deformation algorithm is applied to the left and right processing sub-regions respectively to generate face-slimming image frames.

[0034] The image frames processed by the face slimming method are time-stamped and synchronized with the original audio stream in the video file, and the final video output is generated based on the encoding format in the initialization processing parameters.

[0035] Specifically, this invention acquires the initial attribute information of video files and integrates this static baseline data with the video processing flow to generate initial processing parameters for managing resolution, frame rate, and encoding format. In the frame-by-frame processing stage, facial landmark detection technology is used to accurately identify and extract facial features, transforming dynamic image data into an operable landmark dataset. This dataset is then used as a decision-making basis, intelligently dividing the cheek region into two independent sub-regions (left and right) along the facial midline. Based on the width difference between these two sub-regions, the face-slimming intensity is adaptively adjusted to achieve personalized and differentiated processing for asymmetrical facial shapes. The image frames processed by the local deformation algorithm are time-stamped and synchronized with the original audio stream to ensure that the processed video achieves improved visual effects without audio-visual desynchronization. Based on the encoding format in the initial processing parameters, the processed video frames and audio stream are encapsulated to form a complete, high-quality video output, thus achieving continuous, dynamic, and adaptive face-slimming processing for video faces.

[0036] By employing facial landmark detection and zoning control technology, it achieves refined facial slimming and effectively solves the problem of audio-visual asynchrony in videos by using timestamp synchronization processing. It can provide natural and personalized facial beautification effects while ensuring real-time performance, significantly improving user experience and video processing quality.

[0037] Optionally, the steps of acquiring the video file and parsing the video attribute information, initializing the video processing flow, and generating initialization processing parameters include:

[0038] The video file is checked to verify its existence and read permissions;

[0039] Specifically, a check is performed on the specified video file path to confirm that the file exists and has read permissions. This ensures that subsequent processing can proceed smoothly.

[0040] Read the metadata stream inside the video file and initialize the video processing flow to obtain initial processing parameters including video resolution, frame rate, total number of frames and encoding format.

[0041] Specifically, the metadata stream inside the video file is read to obtain video attribute information, including resolution, frame rate, total number of frames, and encoding format. Based on this attribute information, the video processing flow is initialized, setting the encoding format of the output video to be consistent with the input video, while keeping the frame rate and resolution unchanged, thereby generating initial processing parameters.

[0042] Optionally, the step of reading image frames of the video file frame by frame and performing facial landmark detection on each frame according to the frame rate set in the initialization processing parameters includes:

[0043] The face detection function is used to identify the location of faces in image frames and generate face region data;

[0044] Specifically, when processing the video stream frame by frame, a pre-trained face detection function is first invoked and run on each frame to quickly identify and determine the precise location of faces in the image, typically represented by a rectangular bounding box. This bounding box data contains the coordinate information of the face region, ensuring that subsequent keypoint detection and beautification processing only apply to the face region, rather than the entire image.

[0045] Facial feature points are extracted within the area defined by the face region data to generate a preliminary key point set;

[0046] Specifically, after acquiring the facial region data, an algorithm specifically designed for facial feature point localization is immediately applied within the area defined by the rectangular frame. This algorithm can accurately identify multiple key feature points on the face, such as the centers of the eyes, the tip of the nose, the corners of the mouth, the eyebrows, and the chin contour. This precise set of coordinate points constitutes the initial key point set, providing a highly refined data foundation for subsequent face slimming and segmentation processing.

[0047] The initial key point set is subjected to cross-frame temporal smoothing filtering to suppress inter-frame jitter and generate a facial key point dataset.

[0048] Specifically, to address the issue of keypoint position jitter in videos caused by slight movements of the subject or changes in lighting, a temporal smoothing filter is applied to the initial keypoint set. This involves calculating a weighted average of the keypoint coordinates in the current frame and the coordinates of historical keypoints from previous frames. For example, a decay factor can be set to give higher weight to more recent frames and lower weight to earlier frames, thus suppressing inter-frame jitter while ensuring the smoothness and consistency of keypoint positions, ultimately generating a stable and accurate facial keypoint dataset. Figure 2 As shown, the method of the present invention significantly improves the stability of facial landmark detection and quantifies the effectiveness of cross-frame temporal smoothing filtering.

[0049] Optionally, dividing the facial region into left and right processing sub-regions includes:

[0050] Extract jawline points from the facial key point dataset to generate cheek contour data;

[0051] Specifically, after obtaining the facial landmark dataset, specific coordinate points representing the jawline are precisely extracted from it. These points are typically located on both sides of the chin and cheeks, outlining the overall facial contour. By concatenating these coordinate points, a cheek contour line is formed and stored as cheek contour data, laying the foundation for subsequent facial region segmentation.

[0052] The facial midline is determined based on the nose tip and chin center point in the facial key point dataset, and midline data is generated.

[0053] Specifically, to determine the facial midline, the tip of the nose and the center of the chin are selected from the facial landmark dataset. These two points represent the vertical center line of the face and can be used to calculate a straight line passing through them. Assuming the coordinates of the tip of the nose are... The coordinates of the center point of the chin are The formula for calculating the facial midline L is:

[0054] ,

[0055] The formula can generate central axis data, providing a precise reference line for facial region segmentation.

[0056] The cheek contour data is segmented using the central axis data to generate partitioned configuration data containing the left cheek area and the right cheek area.

[0057] Specifically, after determining the facial midline, this midline is used as the segmentation basis to divide the cheek contour data. All contour points and pixels located to the left of the midline are classified as the left cheek region, while all contour points and pixels located to the right of the midline are classified as the right cheek region. This segmentation method ensures the independence of the left and right cheeks, thus allowing for differentiated face-slimming treatment on both sides. Finally, this partitioning information is integrated to generate partition configuration data.

[0058] Optionally, calculating the width difference between the left and right processing sub-regions and adaptively adjusting the face-slimming intensity includes:

[0059] Based on the partition configuration data, calculate the first width value of the left cheek area and the second width value of the right cheek area to generate width difference data;

[0060] Specifically, after obtaining the partition configuration data, the widths of the left and right cheek regions are calculated. The first width value of the left cheek region is calculated. This represents the horizontal distance from the midline of the face to the furthest point of the left cheek contour. Similarly, the second width value of the right cheek area is calculated. These two width values ​​are used to calculate the width difference data. Its formula is:

[0061] ,

[0062] This difference data quantifies the degree of asymmetry between the left and right sides of the face.

[0063] The width difference data is compared with a preset symmetry threshold to determine the asymmetry of the face shape and generate an asymmetry marker.

[0064] Specifically, the calculated width difference data With the preset symmetry threshold The comparison is performed. This threshold is a configurable parameter that defines the acceptable range of facial symmetry. If the width difference data exceeds this threshold, facial asymmetry is considered to exist, and an asymmetry flag is generated. This logic can be expressed by the following formula:

[0065] ,

[0066] ,

[0067] The value 1 indicates that there is asymmetry, and the value 0 indicates that the face shape is basically symmetrical.

[0068] Based on the asymmetry markers and the user-defined base face-slimming intensity, differentiated and adjusted face-slimming parameters are generated.

[0069] Specifically, if the asymmetry indicator shows facial asymmetry, the user-defined basic face-slimming intensity will be used in conjunction with the adjustment. and regulatory factors The intensity of face-slimming is adjusted differently on the left and right sides. Assume the average width of the left and right cheek areas is... The calculation formula is as follows:

[0070] ,

[0071] So, what are the intensity parameters for slimming the face on the left side? And the intensity parameters for slimming the face on the right side The calculations will be performed according to the following formulas:

[0072] ,

[0073] ,

[0074] This approach uses a higher slimming intensity on the wider side and a lower intensity on the narrower side to achieve the adjusted slimming parameters. For example... Figure 3 As shown in the figure, the line graph compares the effect of the method of the present invention on correcting the width ratio of the left and right cheeks under different head postures, indicating that the adaptive adjustment strategy of the present invention can make the face shape more symmetrical.

[0075] Optionally, based on the asymmetry marker, and in conjunction with the user-defined base face-slimming intensity, the following may be included:

[0076] If the asymmetry indicator indicates that there is facial asymmetry, then the left adjustment factor and the right adjustment factor are calculated based on the width difference data to generate adjustment factor data;

[0077] Specifically, if the asymmetry indicator shows facial asymmetry, a left adjustment factor will be calculated based on the width difference data to adjust the intensity of face slimming. and right-side adjustment factor The calculation of these two adjustment factors is based on a user-defined adjustment factor. The left-side adjustment factor and the right-side adjustment factor are calculated according to the following formulas:

[0078] ,

[0079] ,

[0080] These two formulas generate adjustment factor data, providing specific values ​​for differentiated face slimming intensity adjustments.

[0081] The basic face-slimming intensity is combined with the adjustment factor data to generate the adjusted face-slimming parameters.

[0082] Specifically, the user-defined basic face-slimming intensity is non-linearly superimposed with the left and right adjustment factors to generate the final left and right face-slimming intensity parameters.

[0083] Optionally, applying the local deformation algorithm to the left and right processing sub-regions respectively includes:

[0084] For each processing sub-region, calculate the distance between the pixels within the sub-region and the geometric center of the sub-region, and generate distance weight data;

[0085] Specifically, it iterates through all pixels within each processing sub-region and calculates the Euclidean distance from each pixel to the geometric center of that sub-region. Pixels that are closer to each other should be more affected by the face-slimming deformation; therefore, this distance value is used as the basis for calculating the deformation weight. The distance weight information of all pixels is integrated to generate distance weight data.

[0086] For example, the distance from a pixel to the center of its sub-region can be used to calculate a weight value. This weight can be defined using a function: when the pixel is at the center of the sub-region, the weight is a maximum of 1.0; when the pixel is at the edge of the sub-region, the weight gradually decreases to 0, thus achieving a smooth deformation effect from the inside out.

[0087] By combining the distance weight data with the adjusted face-slimming parameters, a displacement vector is calculated for each pixel to generate displacement mapping data;

[0088] Specifically, based on distance weight data and adjusted face-slimming parameters, a displacement vector is calculated for each pixel. This displacement vector determines the direction and distance the pixel should move during the face-slimming process. This displacement vector is applied to each pixel, and its new coordinate position is recorded, thereby generating displacement mapping data.

[0089] For example, for a pixel located in the left cheek area, its displacement vector points towards the midline of the face, and the displacement distance is determined by its distance weight and the face-slimming parameters adjusted on the left side. Thus, pixels closer to the center move a greater distance.

[0090] Based on the displacement mapping data, pixels are resampled using inverse mapping and bilinear interpolation algorithms to generate image frames with slimmer faces.

[0091] Specifically, to generate the slimmed-down image, a reverse mapping technique is used to iterate through every pixel in the target image and calculate its corresponding position in the original image based on the displacement mapping data. Since this corresponding position is usually not an integer coordinate, a bilinear interpolation algorithm is used to perform a weighted average based on the color values ​​of the four surrounding neighboring pixels to obtain an accurate color value. All these resampled pixels constitute the final slimmed-down image frame.

[0092] For example, if the corresponding position of a target pixel in the original image is (100.5, 200.5), bilinear interpolation is used to calculate the color value of (100.5, 200.5) by weighted average based on the color values ​​of its four surrounding pixels (100, 200), (101, 200), (100, 201), and (101, 201), ensuring that the resampled image is visually smooth and natural without jagged edges or blurriness.

[0093] Optionally, the step of combining the distance weight data with the adjusted face-slimming parameters to calculate the displacement vector for each pixel includes:

[0094] Boundary checks are performed on the calculated displacement vectors to prevent pixel coordinates from exceeding the image range and to generate valid displacement data.

[0095] Specifically, after calculating the displacement vector for each pixel, a boundary check is immediately performed on the new coordinates of the pixel. This check step ensures that after applying the displacement vector, the new position of the pixel remains within the valid range of the image, i.e., it does not exceed the width or height boundaries of the image. Any pixels that exceed the boundaries will be discarded or corrected to prevent image distortion, thereby generating valid displacement data.

[0096] For example, if the new coordinates of a pixel are calculated to be (-10, 50), the x-coordinate is corrected to 0 because it exceeds the left boundary of the image, or the pixel is simply ignored to avoid anomalies in the image processing process.

[0097] Gaussian blurring is applied to the displacement vectors at the edge of the processing sub-region in the effective displacement data to smooth the boundary between the deformed region and the unprocessed region, generating displacement mapping data.

[0098] Specifically, to ensure a smooth and natural transition between the deformed and unprocessed regions, Gaussian blurring is applied to the displacement vectors located at the edges of the processed sub-regions in the effective displacement data. Gaussian blurring uses a Gaussian function as a basis to apply a weighted average to the displacement vectors based on the distance from each pixel to the center of the processed sub-region. This process results in pixels closer to the sub-region edge having smaller displacements, thus creating a smooth transition from strong deformation to no deformation.

[0099] For example, the slimming effect should gradually diminish at the edges of the cheeks, rather than stopping abruptly. Gaussian blur acts like a soft light on the slimming effect, allowing the cheek contour to smoothly transition to the ear or hairline area, avoiding a harsh, one-dimensional slimming effect and thus ensuring a natural look for the enhancement.

[0100] Optionally, the image frames processed by the face slimming procedure are time-stamped and synchronized with the original audio stream in the video file, and the final video output is generated based on the encoding format in the initialization processing parameters, including:

[0101] Extract the playback time of each audio packet from the original audio stream in the video file to generate audio timestamp data;

[0102] Specifically, an audio decoder is used to separate the raw audio stream from the video file, and data packets are read one by one from the audio stream. When reading each audio packet, the playback timing information embedded within it is extracted; this information is typically in the form of a timestamp. All extracted timestamp information is processed to generate audio timestamp data, which serves as a reference for subsequent video frame synchronization.

[0103] For example, an AAC format audio stream is extracted from the original video file. During the parsing of this audio stream, each audio packet carries a DTS and a PTS. The PTS value of each audio packet is extracted and recorded for subsequent synchronization with video frames.

[0104] Match the timestamp in the audio timestamp data to each image frame after face slimming to generate adjusted timestamp data;

[0105] Specifically, after the face-slimming process is completed, a new timestamp is calculated for each processed image frame. This new timestamp is adjusted based on the frame's original timestamp and the actual latency generated during video processing. The audio timestamp data is then iterated through, finding the closest audio timestamp for each image frame to ensure alignment of the image and audio on the timeline. These adjusted timestamps are then recorded.

[0106] For example, suppose the original video frame has a timestamp of 100 milliseconds, but a 20-millisecond delay is introduced during the face-slimming process. Then, a new timestamp, such as 120 milliseconds, will be assigned to this processed image frame and matched with the audio timestamp that is closest to 120 milliseconds.

[0107] Maintain a mapping table that records the correspondence between the image frames after face slimming and the adjusted timestamp data, and use the mapping table during video encapsulation to generate the final video output based on the encoding format in the initialization processing parameters.

[0108] Specifically, a timestamp mapping table is created to record each image frame processed by face slimming and its corresponding adjusted timestamp. This mapping table is a crucial data structure that ensures the correspondence between video frames and audio streams is not lost throughout the processing. In the final video encapsulation stage, an encapsulation component is invoked, referring to this mapping table, to merge the processed video frames with the original audio stream in the correct time order, and generate the final video file according to the encoding format specified in the initial processing parameters, such as MP4 or FLV.

[0109] For example, during the video encapsulation process, the encapsulation component first finds the first frame image and its adjusted timestamp from the mapping table, then finds the first audio packet and its timestamp, and writes both to the output file in chronological order. This process continues until all video frames and audio packets have been correctly written, thus ensuring that the final generated video file is synchronized with the audio.

[0110] Based on the same inventive concept, such as Figure 4 As shown, the present invention also provides a video face slimming system based on key point partitioning control, the system comprising:

[0111] An initialization module is used to acquire video files and parse video attribute information, initialize the video processing flow, and generate initialization processing parameters, including video resolution, frame rate, and encoding format.

[0112] The key point detection module is used to read image frames one by one and perform facial key point detection according to the frame rate set in the initialization processing parameters to generate a facial key point dataset.

[0113] The partitioning module is used to divide the facial region into two processing sub-regions, left and right, based on the facial key point dataset, and generate partitioning configuration data.

[0114] The parameter adjustment module is used to calculate the width difference based on the partition configuration data and adaptively adjust the face slimming intensity to generate the adjusted face slimming parameters;

[0115] The deformation processing module is used to apply a local deformation algorithm to the left and right processing sub-regions respectively using the adjusted face-slimming parameters to generate face-slimming image frames.

[0116] The synchronous output module is used to perform time-stamp synchronization processing on the image frames after face slimming and the original audio stream, and generate the final video output based on the encoding format in the initialization processing parameters.

[0117] Example 1:

[0118] To verify the feasibility of this invention in practice, it was applied to a professional video editing software suite. This software suite aims to provide content creators with efficient and high-quality video enhancement functions. Currently, the software's built-in "classic slimming" filter uses a uniform deformation algorithm, which works reasonably well when processing frontal videos. However, when processing videos where the user's head is slightly turned or their face shape is asymmetrical, it often produces unnatural, stiff, or even distorted visual effects. Furthermore, the jawline after processing has a harsh boundary with the unprocessed area, resulting in low user satisfaction. The software development team hopes to use this invention to achieve a more intelligent, natural, and personalized video slimming effect.

[0119] In this embodiment, the method and system of the present invention are integrated into its new "AI Intelligent Beautification" function module. The user uploads a 30-second selfie video with a resolution of 1920x1080 and a frame rate of 30fps. In the video, the user speaks directly to the camera, accompanied by slight left-right head movements.

[0120] To verify the beneficial effects of the present invention, this embodiment processes the same video using the "classic face slimming" filter and the "AI intelligent beautification" function of the present invention, and records and compares the performance data of key technical aspects and the final user experience.

[0121] In the facial landmark detection step, images are read frame by frame. The old algorithm in the control group directly extracts landmarks. When the user's head rotates, the jawline landmarks exhibit an average jump of 2.3 pixels between adjacent frames, resulting in a barely perceptible "jitter" artifact in the processed jawline contour. The experimental group uses the method of this invention, which, after extracting the initial landmark set, applies cross-frame temporal smoothing filtering to suppress the average jump of landmarks between frames to within 0.5 pixels. This provides excellent temporal continuity for the facial landmark dataset, fundamentally solving the image flickering problem caused by landmark instability in subsequent processing.

[0122] In the facial segmentation and parameter adaptive adjustment step, at the 15-second mark of the video playback, the user's head tilted approximately 15 degrees to the left, making their right cheek appear wider than their left. The control group algorithm did not differentiate between these two angles, applying the same 50% slimming intensity set by the user to both sides. The experimental group, however, dynamically generated a facial midline based on the tip of the nose and the center point of the chin, and used this as a benchmark to precisely segment the cheek contour data. The calculated width of the right cheek area was 120 pixels, while the left was 100 pixels. This width difference exceeded the preset 10% symmetry threshold, generating an asymmetry marker. Based on this marker, the system, combined with the user-set 50% base intensity, calculated and generated differentiated adjusted slimming parameters: an enhanced 55% slimming intensity was applied to the wider right cheek area, while a reduced 45% slimming intensity was applied to the narrower left cheek area.

[0123] In the application of the local deformation algorithm, the control group showed a clear visual boundary at the junction of the cheek and neck. The experimental group, however, demonstrated a significant advantage. First, multiple concentric circular regions with increasing radii were defined for the left and right processing sub-regions, each centered on its geometric center, and deformation coefficients decreasing from the inside out were assigned to ensure a smooth transition of the slimming effect from the center of the cheek to the edge. Second, Gaussian blurring was applied to the displacement vectors at the edges of the processing sub-regions, further softening the boundary between the deformed and unprocessed areas. Finally, in the processed image frame, the slimming effect blended naturally into the surrounding skin without any harsh boundary.

[0124] Finally, in the synchronous output step, due to the higher computational complexity of the "AI Intelligent Beautification" function compared to traditional filters, the processing time per frame fluctuates between 30-40 milliseconds. By accurately extracting the timestamps of the original audio stream, the timestamps of each image frame after face slimming are re-matched and corrected, and the corresponding relationship is recorded in a mapping table. During the final video encapsulation, this mapping table ensures that the processed video image is perfectly synchronized with the user's lip movements and voice, avoiding audio-visual disconnect.

[0125] The technical effectiveness of this invention has been fully verified through objective data analysis and subjective evaluation of the processed video.

[0126] Table 1 Comparison of Keypoint Detection Stability and Jitter Suppression

[0127]

[0128] Table 2 Comparison of asymmetry correction effects under different head postures

[0129]

[0130] Table 3 Comparison of User Subjective Satisfaction Ratings (1-5 points, higher scores are better)

[0131]

[0132] As can be seen from the data in Tables 1 to 3 above, the method of this invention has comprehensive technical advantages in video face slimming processing. The data in Table 1 clearly shows that the cross-frame temporal smoothing filtering processing of this invention greatly improves the stability of facial key points, reducing the average inter-frame displacement by 78.3%, thus ensuring the smoothness and continuity of the face slimming effect from the source. The data in Table 2 demonstrates the breakthrough of this invention in its core function: intelligent asymmetric correction capability. When the user's head is turned, the control group cannot improve the visual width difference of the cheeks, while this invention can adaptively adjust the bilateral face slimming intensity according to the width difference, making the processed cheek width ratio significantly approach a symmetrical 1.0, resulting in a more harmonious visual effect. Finally, as shown in Table 3, these technical advantages directly translate into a significant improvement in user experience. The scores of this invention in terms of effect naturalness, boundary smoothness, and overall visual effect are all far higher than traditional methods, gaining widespread user recognition.

[0133] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method can be applied to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0134] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A video face slimming method based on key point partitioning control, characterized in that, The method includes: The video file is acquired and its attribute information is parsed. The video processing flow is initialized and initialization processing parameters are generated, including video resolution, frame rate and encoding format. The video file is read frame by frame, and facial landmark detection is performed on each frame according to the frame rate set in the initialization processing parameters to generate a facial landmark dataset. Based on the aforementioned facial key point dataset, the facial region is divided into two processing sub-regions, left and right, to generate partition configuration data; Based on the partition configuration data, the width difference between the left and right processing sub-regions is calculated, and the face-slimming intensity is adaptively adjusted to generate adjusted face-slimming parameters. The calculation of the width difference between the left and right processing sub-regions and the adaptive adjustment of the face-slimming intensity includes: calculating a first width value for the left cheek region and a second width value for the right cheek region based on the partition configuration data to generate width difference data; comparing the width difference data with a preset symmetry threshold to determine the asymmetry of the face shape and generating an asymmetry indicator; and generating differentiated adjusted face-slimming parameters based on the asymmetry indicator and the user-defined base face-slimming intensity. Using the adjusted face-slimming parameters, a local deformation algorithm is applied to the left and right processing sub-regions respectively to generate face-slimming image frames. The image frames after face slimming are time-stamped and synchronized with the original audio stream in the video file, and the final video output is generated based on the encoding format in the initialization processing parameters. The process, based on the asymmetry indicator and the user-defined basic face-slimming intensity, includes: if the asymmetry indicator indicates a face shape asymmetry, then calculating the left and right adjustment factors based on the width difference data to generate adjustment factor data; and combining the basic face-slimming intensity with the adjustment factor data to generate adjusted face-slimming parameters.

2. The video face slimming method based on key point partitioning control according to claim 1, characterized in that, The steps of acquiring video files, parsing video attribute information, initializing the video processing flow, and generating initialization processing parameters include: The video file is checked to verify its existence and read permissions; Read the metadata stream inside the video file and initialize the video processing flow to obtain initial processing parameters including video resolution, frame rate, total number of frames and encoding format.

3. The video face slimming method based on key point partitioning control according to claim 1, characterized in that, The step of reading image frames from the video file frame by frame and performing facial landmark detection on each frame according to the frame rate set in the initialization processing parameters includes: The face detection function is used to identify the location of faces in image frames and generate face region data; Facial feature points are extracted within the area defined by the face region data to generate a preliminary key point set; The initial key point set is subjected to cross-frame temporal smoothing filtering to suppress inter-frame jitter and generate a facial key point dataset.

4. The video face slimming method based on key point partitioning control according to claim 1, characterized in that, The process of dividing the facial region into left and right processing sub-regions includes: Extract jawline points from the facial key point dataset to generate cheek contour data; The facial midline is determined based on the nose tip and chin center point in the facial key point dataset, and midline data is generated. The cheek contour data is segmented using the central axis data to generate partition configuration data containing the left cheek area and the right cheek area.

5. A video face slimming method based on key point partitioning control according to claim 1, characterized in that, The application of the local deformation algorithm to the left and right processing sub-regions includes: For each processing sub-region, calculate the distance between the pixels within the sub-region and the geometric center of the sub-region, and generate distance weight data; By combining the distance weight data with the adjusted face-slimming parameters, a displacement vector is calculated for each pixel to generate displacement mapping data; Based on the displacement mapping data, pixels are resampled using inverse mapping and bilinear interpolation algorithms to generate image frames with slimmer faces.

6. A video face slimming method based on key point partitioning control according to claim 5, characterized in that, The step of combining the distance weight data and the adjusted face-slimming parameters to calculate the displacement vector for each pixel includes: Boundary checks are performed on the calculated displacement vectors to prevent pixel coordinates from exceeding the image range and to generate valid displacement data. Gaussian blurring is applied to the displacement vectors at the edge of the processing sub-region in the effective displacement data to smooth the boundary between the deformed region and the unprocessed region, generating displacement mapping data.

7. The video face slimming method based on key point partitioning control according to claim 1, characterized in that, The image frames processed by the face slimming procedure are time-stamped and synchronized with the original audio stream in the video file. Based on the encoding format in the initialization processing parameters, the final video output is generated, including: Extract the playback time of each audio packet from the original audio stream in the video file to generate audio timestamp data; Match the timestamp in the audio timestamp data to each image frame after face slimming to generate adjusted timestamp data; Maintain a mapping table that records the correspondence between the image frames after face slimming and the adjusted timestamp data, and use the mapping table during video encapsulation to generate the final video output based on the encoding format in the initialization processing parameters.

8. A video face-slimming system based on key point partitioning control, characterized in that, The system is used in a video face slimming method based on key point partitioning control as described in any one of claims 1-7, the system comprising: An initialization module is used to acquire video files and parse video attribute information, initialize the video processing flow, and generate initialization processing parameters, including video resolution, frame rate, and encoding format. The key point detection module is used to read image frames one by one and perform facial key point detection according to the frame rate set in the initialization processing parameters to generate a facial key point dataset. The partitioning module is used to divide the facial region into two processing sub-regions, left and right, based on the facial key point dataset, and generate partitioning configuration data. The parameter adjustment module is used to calculate the width difference and adaptively adjust the face-slimming intensity based on the partition configuration data, generating adjusted face-slimming parameters. The calculation of the width difference between the left and right processing sub-regions and the adaptive adjustment of the face-slimming intensity includes: calculating a first width value for the left cheek region and a second width value for the right cheek region based on the partition configuration data, generating width difference data; comparing the width difference data with a preset symmetry threshold to determine the face shape asymmetry and generating an asymmetry marker; and generating differentiated adjusted face-slimming parameters based on the asymmetry marker and the user-defined base face-slimming intensity. The deformation processing module is used to apply a local deformation algorithm to the left and right processing sub-regions respectively using the adjusted face-slimming parameters to generate face-slimming image frames. The synchronous output module is used to perform time-stamp synchronization processing on the image frames after face slimming and the original audio stream, and generate the final video output based on the encoding format in the initialization processing parameters; The process, based on the asymmetry indicator and the user-defined basic face-slimming intensity, includes: if the asymmetry indicator indicates a face shape asymmetry, then calculating the left and right adjustment factors based on the width difference data to generate adjustment factor data; and combining the basic face-slimming intensity with the adjustment factor data to generate adjusted face-slimming parameters.

Images