A video skin beautification method and system fusing multi-color space and filtering

Abstract

The application discloses a kind of fusion multi-color space and filtering video beautifying method and system, belong to computer vision and digital image processing technical field, it includes real-time analysis original video stream, separates audio and video data;Adopt mixed detection to carry out face detection and landmark calibration;Based on landmark set, construct the mask of protecting five organs, and realize natural contour remodeling by global-local double-center complex deformation field;Optimize light color to skin color in a variety of orthogonal color spaces;Image is decomposed into base layer and detail layer, and high-fidelity skin is realized by self-adaptive edge-preserving filtering to base layer;Processed image sequence and original audio are synchronously encapsulated as standard video file.The application can efficiently solve the problems such as unnatural beauty effect, high processing delay and audio-visual asynchronization in the prior art through full-link collaborative optimization, and significantly improves the realism, real-time performance and stability of video beauty.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to a video beautification method and system that integrates multiple color spaces and filtering. Background Technology

[0002] With the convergence of digital multimedia technology and mobile internet, video has become a primary medium for information dissemination and social interaction. Driven by applications such as short video platforms, live streaming, and video conferencing systems, users are increasingly demanding portrait enhancement features in video calls and content creation. Video beautification technology, as a key element in enhancing the user's visual experience, integrates multiple technologies such as face detection, image enhancement, and computer graphics. It aims to perform real-time beautification of portraits in videos and has become an indispensable core function in various video applications.

[0003] However, existing video beautification technologies still have many shortcomings. In dynamic scenes, beautification effects often become unstable or exhibit edge distortion, and are prone to audio-visual desynchronization, affecting user experience. Specifically, current technologies generally face challenges in the accuracy and robustness of face detection, and sometimes fail under complex lighting or occlusion conditions. Furthermore, face slimming and other contour processing techniques can easily lead to facial distortion and background aberration, while functions such as whitening and skin smoothing often suffer from skin tone distortion and blurred details due to insufficient coordination and optimization, resulting in a stiff and unnatural overall beautification effect. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a video beautification method and system that integrates multiple color spaces and filtering. Through end-to-end collaborative optimization, it can efficiently solve problems such as unnatural beautification effects, high processing latency, and audio-visual asynchrony in existing technologies, significantly improving the realism, real-time performance, and stability of video beautification.

[0005] The above objectives can be achieved through the following approach:

[0006] A video beautification method integrating multiple color spaces and filtering includes real-time reception and reading of raw video streams; buffering and separating the raw video streams to output video data packets and audio data packets; decoding the video data packets frame by frame to obtain unstructured image frames; aligning and encapsulating the unstructured image frames and audio data packets according to timestamps to generate a frame object queue; detecting face regions and evaluating region confidence for the frame object queue; automatically initiating robust compensation detection when the region confidence is lower than a preset face threshold; and performing beautification within the face region. Facial feature points are calibrated to generate a structured landmark set, which is then injected into the frame object queue to form an enhanced frame object queue. Based on the structured landmark set, the facial region is segmented into a rigid protection region and a non-rigid deformation region. A basic deformation field and a local correction field are calculated in the non-rigid deformation region. The basic deformation field and the local correction field are then vector-weighted and synthesized to generate a composite deformation flow field. The original pixel grid of the enhanced frame object queue is resampled according to the composite deformation flow field to output a contour reshaping image. The contour reshaping image is then projected in parallel onto an orthogonal color feature set. The orthogonal color feature space includes a luminance / chrominance space and a perceptual uniformity space. Within the luminance / chrominance space, hue remapping is performed on the luminance channel, and a compensation transformation is applied to the chrominance channel. Within the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted, fused, and inversely transformed back to the standard color space to output a reconstructed light and color image. The reconstructed light and color image is decomposed into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the local facial regions of the structured landmark set, [further details should be added]. Nonlinear spatial domain joint filtering is used to filter out imperfections while preserving edge awareness, resulting in an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output a skin-smoothing and reshaped image. The video encoder is initialized based on the metadata in the enhanced frame object queue. The skin-smoothing and reshaped image sequence is then fed into the video encoder frame by frame in timestamp order to generate a pure video basic stream. Using the timestamp of the audio data packet as a reference, the pure video basic stream and the audio data packet are encapsulated using multiplexing technology to output beautified video data.

[0007] Optionally, the generated frame object queue includes: real-time buffering and synchronous parsing of the original video stream, deconstructing it into a video data packet sequence and an audio data packet sequence; frame-by-frame decoding of the video data packet sequence, extracting pixel information and timestamps to form an unstructured image frame sequence; using the timestamp of the audio data packet sequence as a synchronization reference, temporally aligning the unstructured image frame sequence with the audio data packet sequence, and encapsulating the alignment result into a metadata frame object; organizing the metadata frame objects in chronological order and storing them in a first-in-first-out cache to construct a frame object queue.

[0008] Optionally, forming the enhanced frame object queue includes: extracting frame objects from the frame object queue, detecting statistical features of local gradient distribution in the image, generating a first face region candidate box and evaluating the region confidence score; if it is determined that the region confidence score does not reach the face threshold, then automatically activating robust compensation detection for secondary detection to generate a second face region candidate box; standardizing the format of the first or second face region candidate box and performing high-density facial feature point calibration to generate a structured landmark set; injecting the structured landmark set as enhancement data into the frame objects, and reorganizing all the frame objects to form the enhanced frame object queue.

[0009] Optionally, the output contour reshaping image includes: segmenting the facial region into rigid protection regions and non-rigid deformation regions based on the topological relationship of the structured landmark set; applying Gaussian smoothing filtering to the non-rigid deformation regions to generate a smooth weight mask; calculating a basic deformation field with the nose tip landmark as the geometric center and a local correction field with the chin center landmark as the geometric center within the non-rigid deformation regions based on the smooth weight mask; generating a composite deformation flow field by vector weighted synthesis of the basic deformation field and the local correction field; and performing reverse mapping resampling on the original pixel mesh in the enhanced frame object according to the composite deformation flow field to output the contour reshaping image.

[0010] Optionally, the method further includes: performing geometric size analysis on the first face region candidate box or the second face region candidate box, calculating the facial image scale, and obtaining facial scale parameters; performing vector amplitude statistical analysis on the composite deformation flow field, quantifying the contour reshaping intensity, and obtaining a deformation intensity index; and nonlinearly fusing the facial scale parameters and the deformation intensity index to generate an adaptive control parameter set.

[0011] Optionally, the output color reshaping image includes: within the luminance-chroma space, enhancing dark areas and suppressing highlights in the luminance channel, while reducing the red component and increasing the yellow component in the chroma channel to generate first optimized color data; within the perceptually uniform space, enhancing the gain of the luminance channel, while reducing the red component and increasing the yellow component in the chroma channel to generate second optimized color data; weighting and synthesizing the first optimized color data and the second optimized color data to generate preliminary fused color data; transforming the preliminary fused color data to a universal color space, and globally fine-tuning the saturation and luminance channels to balance the overall tone to obtain globally fine-tuned color data; and inversely transforming the globally fine-tuned color data back to the standard color space to output the color reshaping image.

[0012] Optionally, the output skin smoothing and reshaping image includes: based on the light and color reshaping image, separating basic light and shadow information and color information to obtain a low-frequency base layer, and separating surface texture and minute details to obtain a high-frequency detail layer; dynamically setting the nonlinear spatial kernel size and color similarity weight according to the adaptive control parameter set to obtain a joint filter; selectively smoothing the low-frequency base layer according to the joint filter to filter out imperfections and maintain edge awareness, generating an optimized low-frequency base layer; and performing layered recombination of the optimized low-frequency base layer and the high-frequency detail layer through linear superposition to output the skin smoothing and reshaping image.

[0013] Optionally, the optimized low-frequency base layer includes a smooth color component and an edge-preserving contour component, wherein: the smooth color component is used to present the base skin tone information after blemish suppression and color equalization; the edge-preserving contour component is used to preserve the high-frequency edge information that defines the facial structure and light and shadow boundaries and is identified and protected in the smoothing process.

[0014] Optionally, the output beautified video data includes: setting target resolution and frame rate parameters based on the metadata recorded in the enhanced frame object queue, and initializing the video encoder; submitting the skin smoothing and reshaping image sequence frame by frame to the video encoder according to the timestamp order in the enhanced frame object queue to generate a pure video basic stream; using the timestamp of the audio data packet as a synchronization reference, multiplexing the pure video basic stream and the audio data packet, encapsulating them into a digital media container format, generating beautified video data, and outputting it to the video terminal.

[0015] Based on the same inventive concept, this invention also provides a video beautification system integrating multiple color spaces and filtering. The system includes: a video preprocessing module for receiving and reading raw video streams in real time, buffering and separating the raw video streams, outputting video data packets and audio data packets, decoding the video data packets frame by frame to obtain unstructured image frames, aligning and encapsulating the unstructured image frames and the audio data packets according to timestamps to generate a frame object queue; and a face detection module for detecting face regions and evaluating region confidence scores within the frame object queue, and automatically detecting face regions when the region confidence score is lower than a preset face threshold. The robust compensation detection is initiated automatically. Within the face region, facial feature points are calibrated to generate a structured landmark set, which is then injected into the frame object queue to form an enhanced frame object queue. The contour reshaping module, based on the structured landmark set, segments the face region into rigid protection regions and non-rigid deformation regions. A base deformation field and a local correction field are calculated within the non-rigid deformation region. These are then vector-weightedly synthesized to generate a composite deformation flow field. The original pixel grid of the enhanced frame object queue is resampled based on this composite deformation flow field to output the contour reshaping image. The light and color reshaping module uses… The contour-reconstructed image is projected in parallel onto an orthogonal color feature space, which includes a luminance / chrominance space and a perceptual uniformity space. In the luminance / chrominance space, hue remapping is performed on the luminance channel, and a compensation transformation is applied to the chrominance channel. In the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted, fused, and inversely transformed back to the standard color space to output the light-color reconstructed image. The skin-smoothing and reconstructing module decomposes the light-color reconstructed image into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the structured landmarks... The local facial region of the image is processed by nonlinear spatial domain joint filtering to remove blemishes while maintaining edge awareness, resulting in an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output a skin-smoothing and reshaped image. The video reconstruction module initializes the video encoder based on the metadata in the enhanced frame object queue, sends the skin-smoothing and reshaped image sequence frame by frame into the video encoder in time-stamp order to generate a pure video basic stream, and encapsulates the pure video basic stream and the audio data packet using multiplexing technology based on the timestamp of the audio data packet to output beautified video data.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] 1. Employing a "global-local" dual-center composite deformation model, this approach combines overall contraction centered on the nose tip with independent refinement centered on the chin, avoiding the stiff or disproportionate chin lines caused by traditional single-center deformation. Simultaneously, by constructing and smoothly eliminating contour masks that exclude facial features, it fundamentally prevents distortion of key features, achieving a highly natural and harmonious contour reshaping effect across various face shapes.

[0018] 2. By synergistically utilizing the advantages of multiple orthogonal color spaces, brightness and chroma are adjusted in parallel in a multi-dimensional and differentiated manner. This method not only precisely enhances skin tone whiteness but also avoids local overexposure or pale, distorted skin tones through global tone balancing. By fusing the processing results of different color spaces, a more delicate and layered natural light and color reshaping effect is achieved compared to adjustments in a single color space.

[0019] 3. The image is decomposed into a low-frequency base layer and a high-frequency detail layer. Only the low-frequency layer containing blemishes is smoothed using an edge-preserving filter. This layered optimization strategy efficiently filters out blemishes such as pigmentation while completely preserving the skin's inherent high-frequency texture details. Through layered reconstruction, a perfect balance between even and delicate skin texture and high-fidelity preservation of real details is achieved, effectively avoiding the "mask-like" and blurry appearance of traditional skin-smoothing techniques.

[0020] 4. A complete, end-to-end processing pipeline has been constructed, from real-time video stream parsing and audio-visual synchronization separation to multi-stage asynchronous beautification processing and finally standardized video encapsulation. This process ensures the stability of the front-end input through a robust face detection strategy, and guarantees excellent real-time performance and stability of the entire beautification solution under high-resolution video by supporting GPU acceleration and multi-threaded processing, which can meet the demanding application scenarios such as live streaming and video conferencing.

[0021] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1This is a flowchart illustrating a video beautification method that integrates multiple color spaces and filtering according to an embodiment of the present invention.

[0024] Figure 2 This is a distribution diagram of the dual-center composite deformation intensity along the facial contour according to an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of nonlinear hue remapping of the luminance channel according to an embodiment of the present invention.

[0026] Figure 4 This is a schematic diagram of surface texture optimization based on frequency layering according to an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of a video skin-beautifying system that integrates multiple color spaces and filtering according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Reference Figure 1 One embodiment of the present invention proposes a video beautification method that integrates multiple color spaces and filtering. Through collaborative optimization across the entire process, it can efficiently solve problems such as unnatural beautification effects, high processing latency, and audio-visual asynchrony in existing technologies, and significantly improve the realism, real-time performance, and stability of video beautification.

[0030] The method described in this embodiment specifically includes:

[0031] The system receives and reads the raw video stream in real time, buffers and separates the raw video stream, outputs video data packets and audio data packets, decodes the video data packets frame by frame to obtain unstructured image frames, aligns and encapsulates the unstructured image frames and the audio data packets according to the timestamp, and generates a frame object queue.

[0032] For the frame object queue, the face region is detected and the region confidence is evaluated. When the region confidence is lower than the preset face threshold, robust compensation detection is automatically started. Within the face region, facial feature points are calibrated, a structured landmark set is generated, and injected into the frame object queue to form an enhanced frame object queue.

[0033] Based on the structured landmark set, the facial region is segmented into a rigid protection region and a non-rigid deformation region. A basic deformation field and a local correction field are calculated in the non-rigid deformation region. The basic deformation field and the local correction field are vector-weighted and synthesized to generate a composite deformation flow field. The original pixel grid of the enhanced frame object queue is resampled according to the composite deformation flow field to output a contour reshaping image.

[0034] The contour-reconstructed image is projected in parallel onto an orthogonal color feature space, which includes a luminance-chrominance space and a perceptual uniformity space. In the luminance-chrominance space, hue remapping is performed on the luminance channel and a compensation transformation is applied to the chrominance channel. In the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted and fused and inversely transformed back to the standard color space to output a light and color reconstructed image.

[0035] The light and color reconstructed image is decomposed into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the local facial region of the structured landmark set, a nonlinear spatial domain joint filtering process is applied to filter out imperfections and maintain edge awareness, resulting in an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output a skin smoothing and reconstructed image.

[0036] The video encoder is initialized based on the metadata in the enhanced frame object queue. The skin smoothing and reshaping image sequence is sent to the video encoder frame by frame in time stamp order to generate a pure video basic stream. Based on the timestamp of the audio data packet, the pure video basic stream and the audio data packet are encapsulated using multiplexing technology to output beautified video data.

[0037] Through collaborative optimization across the entire process, problems such as unnatural beautification effects, high processing latency, and audio-visual asynchrony in existing technologies can be efficiently solved, significantly improving the realism, real-time performance, and stability of video beautification.

[0038] Optionally, the queue of generated frame objects includes:

[0039] The original video stream is buffered and parsed in real time, deconstructing it into a video data packet sequence and an audio data packet sequence;

[0040] Specifically, a multimedia processing framework, such as OpenCV or MoviePy, is used to read and decode the received raw video data. This process aims to parse the video file or video stream into a data sequence containing video frames and a separate audio data sequence, providing a foundation for subsequent audio-visual separation processing. This step supports various common video container formats, such as MP4, MOV, or AVI, and various audio formats, such as MP3 or WAV, to ensure good compatibility with data from different devices and acquisition sources.

[0041] The video data packet sequence is decoded frame by frame to extract pixel information and timestamps, forming an unstructured image frame sequence;

[0042] Specifically, through decoding, each frame in the video data packet sequence is converted into raw image data containing pixel information. Simultaneously, metadata for each frame in the original video is extracted and recorded. This metadata includes, but is not limited to, the original video's frame rate, resolution, total number of frames, and precise timestamp information. This information is crucial for subsequent synchronization and reconstruction.

[0043] Using the timestamp of the audio data packet sequence as a synchronization reference, the unstructured image frame sequence is time-aligned with the audio data packet sequence, and the alignment result is encapsulated into a metadata frame object;

[0044] Specifically, to ensure audio-visual synchronization in the final output video, this step is based on the timing of the audio stream. Using a framework such as MoviePy, the audio tracks in the original video file are precisely separated to obtain the complete audio data stream. Subsequently, each unstructured image frame is associated with and encapsulated with its corresponding audio data segment at that time point, forming a structured metadata frame object. This object contains both the pixel content of the image and synchronization information to ensure timing consistency.

[0045] The metadata frame objects are organized in chronological order and stored in a first-in-first-out cache to construct a frame object queue.

[0046] Specifically, multiple encapsulated metadata frame objects are arranged according to their original timestamp order and sent to a First-In, First-Out (FIFO) data buffer queue. This queue serves as the data source for subsequent processing modules, ensuring that image frames are processed in the correct order and stably. This queue-based architecture also provides system-level support for multi-threaded asynchronous execution and GPU (Graphics Processing Unit) acceleration, thereby improving the efficiency and real-time performance of the entire beautification processing workflow.

[0047] Optionally, the formation of the enhanced frame object queue includes:

[0048] Frame objects are extracted from the frame object queue, statistical features of local gradient distribution in the image are detected, a first face region candidate box is generated, and the region confidence score is evaluated.

[0049] Specifically, a master detector is used to perform face detection on image frames extracted from the frame object queue. In a preferred embodiment, this master detector can be implemented based on the Dlib (Dlib C++ Library) framework. This detector extracts features by calculating the Histogram of Oriented Gradients (HOG) of the image and uses a linear Support Vector Machine (SVM) classifier to achieve accurate localization and detection of face regions. After detection, one or more candidate bounding boxes for face regions are output, and a confidence score is calculated for each candidate bounding box to evaluate the reliability of the detection results.

[0050] If the confidence score of the region does not reach the face threshold, robust compensation detection is automatically activated to perform secondary detection and generate a second face region candidate box.

[0051] Specifically, to improve the detection success rate in complex scenes, this method includes a compensation detection mechanism. The face threshold is a pre-set value used to determine the reliability of the main detector's results. When the confidence score output by the main detector is lower than this face threshold, or when the main detector fails to detect any faces, the processing flow automatically activates a more robust compensation detector. In a preferred embodiment, this compensation detector can be a cascaded detector from OpenCV. This detector has a high recall rate when processing large-angle profile faces, partially occluded or blurred video frames, and can generate a candidate second face region bounding box.

[0052] The first or second face region candidate box is formatted and high-density facial feature point calibration is performed to generate a structured landmark set.

[0053] Specifically, to ensure consistency in subsequent processing, the format of the candidate face regions output by the detector needs to be standardized. For example, when the compensation detector is enabled, the coordinates of the rectangular regions output by the cascade detector can be converted to a rectangular format compatible with the Dlib framework. After obtaining the standardized facial regions, a facial landmark detection algorithm is applied within these regions to perform high-density facial landmark calibration. This calibration process aims to accurately extract multiple key coordinate points including facial contours, eyes, nose, mouth, etc., forming a structured landmark set containing facial topological relationships, providing an accurate data foundation for subsequent geometric deformation and region segmentation.

[0054] The structured landmark set is injected into the frame object as enhanced data, and all the frame objects are reorganized to form an enhanced frame object queue.

[0055] Specifically, the structured landmark set is added or updated as a new data field to its corresponding metadata frame object. After this operation, each frame object not only contains the original image, audio, and temporal information, but also precise facial geometry information. Once all frame objects in the queue have completed the landmark set injection, an enhanced frame object queue with enhanced data content is formed. This queue will serve as the input for all subsequent beautification processing steps.

[0056] Optionally, the output contour reconstructed image includes:

[0057] Based on the topological relationship of the structured landmark set, the facial region is segmented into a rigid protection region and a non-rigid deformation region, and a Gaussian smoothing filter is applied to the non-rigid deformation region to generate a smooth weight mask.

[0058] Specifically, this step first generates a binary mask based on the structured landmark set from the previous embodiment. Specifically, a closed polygonal region is constructed by connecting key points of the facial contour, and this region is filled with valid region values, such as 1. Simultaneously, polygonal regions formed by key points corresponding to the eyes, nose, mouth, etc., are filled with invalid region values, such as 0, thereby segmenting the facial region into a rigid protective region that does not participate in geometric transformation and a non-rigid deformation region that does participate in geometric transformation. To ensure a natural transition of subsequent deformation effects at the edges, the binary mask needs to be smoothed. In a preferred embodiment, Gaussian smoothing filtering can be used to generate the final smooth weighted mask. This filtering process can be performed using Gaussian convolution operations on the GPU to improve efficiency. The Gaussian kernel function can be expressed as:

[0059] ,

[0060] in, and It is the distance from the current point to the center of the core; The standard deviation is the value of the face. This standard deviation parameter can be dynamically adjusted according to the size of the face contour to accommodate different face sizes.

[0061] Based on the smooth weight mask, within the non-rigid deformation region, the basic deformation field is calculated with the nose tip landmark as the geometric center, and the local correction field is calculated with the chin center landmark as the geometric center.

[0062] Specifically, this step involves two deformation fields computed in parallel.

[0063] The base deformation field is used to achieve an inward contraction of the overall facial contour. This calculation uses the nose tip coordinates, calibrated in a structured landmark set, as the geometric center and calculates the displacement vector of each pixel within the non-rigid deformation region based on a radial contraction model. The local correction field focuses on a more refined and natural optimization of the chin region to prevent insufficient deformation or an abrupt appearance. This calculation uses the chin center coordinates, calibrated in a structured landmark set, as the geometric center and also employs a radial contraction model, but uses a set of independent intensity and transition parameters to enhance the symmetrical contraction effect of the chin region. The radial contraction model can be expressed as:

[0064] ,

[0065]

[0066] in, It is the calculated pixel displacement vector; These are the coordinates of the geometric center, i.e., the coordinates of the tip of the nose or the center of the chin; It is the Euclidean distance from the current pixel to the geometric center; It is the tensile strength coefficient that controls deformation strength; It is a transition parameter that controls the deformation range. An independent tensile strength coefficient is used when calculating the local correction field. and transition parameters .

[0067] A composite deformation flow field is generated by vector weighting the basic deformation field and the local correction field.

[0068] Specifically, the basic deformation field and the local correction field calculated in the above steps are merged to generate a final composite deformation flow field. This synthesis process is a vector superposition; for each pixel within the non-rigid deformation region, its final displacement vector is the sum of the basic deformation field vector and the local correction field vector. This composite deformation flow field completely records the precise displacement information of each pixel that needs to move in both the horizontal and vertical directions.

[0069] Based on the composite deformation flow field, the original pixel grid in the enhanced frame object is reverse-mapped and resampled to output a contour-reconstructed image.

[0070] Specifically, this step applies an image remapping algorithm to complete the final geometric deformation. In a preferred embodiment, a reverse mapping and interpolation method can be used. For each pixel grid point in the output image, its source coordinate position in the original image is calculated based on the composite deformation flow field. Since these source coordinates are usually non-integer, an image interpolation algorithm, such as bilinear interpolation, is needed to calculate the precise pixel value of the point based on the pixel values ​​of its four surrounding pixels, ultimately generating a contour-reconstructed image with a natural and smooth contour, such as... Figure 2 The diagram shows the distribution of the intensity of the dual-center composite deformation field along the facial contour, illustrating the respective contributions of the base field and the local correction field, as well as the final composite intensity, embodying the "global + local" composite face-slimming concept of this invention.

[0071] Optionally, the method further includes:

[0072] Perform geometric size analysis on the first or second face region candidate box, calculate the facial image scale, and obtain facial scale parameters.

[0073] Specifically, this step analyzes candidate bounding boxes for face regions. This analysis aims to quantify the relative size of the face within the current video frame. In a preferred embodiment, the pixel area or diagonal length of the candidate bounding box can be calculated and compared with the total pixel area or total diagonal length of the current image frame to obtain a standardized ratio. This ratio is the facial scale parameter, which objectively reflects the distance or size of the face in the image, providing a basis for adaptive parameter adjustments in subsequent processing steps.

[0074] The vector amplitude statistical analysis of the composite deformation flow field was performed to quantify the profile reshaping intensity and obtain the deformation intensity index.

[0075] Specifically, this step analyzes the complex deformation flow field. This analysis aims to quantify the intensity of the contour reshaping operation just performed. In a preferred embodiment, a single numerical value can be obtained by calculating the average amplitude or root mean square value of the amplitudes of all displacement vectors in the complex deformation flow field. This value is the deformation intensity index, which objectively reflects the overall intensity of the face-slimming operation; a larger deformation intensity index value indicates a more significant deformation.

[0076] The facial scale parameters and the deformation intensity index are nonlinearly fused to generate an adaptive control parameter set.

[0077] Specifically, this step aims to generate a set of control parameters that can dynamically guide subsequent beautification modules. Using a preset nonlinear mapping function or a small lookup table, the facial scale parameters and deformation intensity indices obtained in the previous two steps are merged into one or more specific parameter values, forming an adaptive control parameter set. This parameter set provides a basis for dynamic adjustment in subsequent filtering processes. For example, a larger facial scale parameter or a higher deformation intensity index may map to a set of filtering parameters requiring a stronger smoothing effect, thereby achieving intelligent and adaptive beautification processing.

[0078] Optionally, the output light and color reconstructed image includes:

[0079] Within the luminance-chrominance space, the luminance channel enhances dark areas and suppresses highlights, while the chrominance channel reduces the red component and increases the yellow component to generate first optimized color data.

[0080] Specifically, in a preferred embodiment, the facial region in the contour-reconstructed image can be converted from the RGB color space to the YCrCb color space, which is a luminance-chrominance space. Within this space, a locally enhanced luminance balancing algorithm is applied to the luminance component. By analyzing the image histogram, the brightness of dark areas is increased and the brightness of highlight areas is suppressed, thereby enhancing the whiteness of skin tone and avoiding overexposure. Simultaneously, the chrominance component is adjusted by reducing the red component and increasing the yellow component to make the skin tone appear whiter.

[0081] Within the perceived uniform space, the gain of the luminance channel is increased, while the red component of the chroma channel is reduced and the yellow component is increased to generate second optimized color data.

[0082] Specifically, in another parallel processing path, the facial area can be converted from the RGB color space to the LAB color space, which is a perceptually uniform space. Within this space, the luminance channel is boosted, and the chroma channel is adjusted, also aiming to reduce the red component and increase the yellow component, in order to achieve a skin tone enhancement effect that conforms to human visual perception.

[0083] The first optimized color data and the second optimized color data are weighted and synthesized to generate preliminary fused color data;

[0084] Specifically, to leverage the advantages of different color space optimizations, the color results generated in the previous two steps, processed in the YCrCb and LAB color spaces respectively, are fused according to a preset weight. This weighted synthesis balances the characteristics of different processing paths, resulting in a preliminary, better-quality fused color result—that is, preliminary fused color data.

[0085] The preliminary fused color data is transformed to a universal color space, and the saturation and brightness channels are globally fine-tuned to balance the overall tone, resulting in globally fine-tuned color data.

[0086] Specifically, to eliminate potential color differences caused by localized whitening and to balance the overall tone, this step performs a global optimization of the initially fused color data. In a preferred embodiment, the data can be transformed to a color model with hue, saturation, and value as basic components, such as the HSV color space. Within this space, the saturation (S channel) and value (V channel) of the entire image are finely adjusted.

[0087] The globally fine-tuned color data is inversely transformed back to the standard color space to output a light and color reconstructed image.

[0088] Specifically, after completing all color enhancement and adjustment steps, the final globally fine-tuned color data is inversely transformed from its original color space back to a standard color space used for device display, such as the RGB color space. This step yields a final output result that combines whiteness, naturalness, and overall tonal balance—a light-reconstructed image, such as... Figure 3 The diagram shown illustrates the nonlinear tone remapping of the luminance channel according to the present invention. It shows the histogram distribution changes of pixel luminance before and after processing, as well as the applied S-shaped luminance mapping curve, demonstrating the effect of brightening dark areas and suppressing highlights.

[0089] Optionally, the output skin smoothing and reshaping image includes:

[0090] Based on the light and color reconstructed image, the basic light and shadow and color information are separated to obtain a low-frequency basic layer, and the surface texture and minute details are separated to obtain a high-frequency detail layer.

[0091] Specifically, this step aims to separate information of different frequencies in the image for differential processing. In a preferred embodiment, this separation process can be achieved by applying an edge-preserving filter, such as a bilateral filtering algorithm. The reconstructed image is input into this filter, and its output is a smooth, low-frequency base layer containing only low-frequency information such as basic lighting and color. Subsequently, by subtracting the original reconstructed image from this low-frequency base layer, a detail layer containing only high-frequency information such as skin texture and pores can be obtained.

[0092] Based on the adaptive control parameter set, the nonlinear spatial kernel size and color similarity weight are dynamically set to obtain a joint filter;

[0093] Specifically, this step prepares the filter for subsequent smoothing processing. In a preferred embodiment, the joint filter used is a bilateral filter. The two core parameters of this filter, namely the spatial kernel size and the color similarity weight, are not fixed. Their specific values ​​are dynamically set based on an adaptive control parameter set. For example, a larger facial scale or stronger deformation intensity reflected in the adaptive control parameter set may be mapped to a set of parameters with a larger spatial kernel size or higher color similarity weight.

[0094] Based on the joint filter, the low-frequency base layer is selectively smoothed to filter out imperfections and maintain edge awareness, thereby generating an optimized low-frequency base layer.

[0095] Specifically, the joint filter, with parameters set in the previous step, is applied only to the low-frequency base layer. Since the high-frequency texture information has been separated into the high-frequency detail layer, the filtering operation here can focus more on smoothing large-scale imperfections in the low-frequency base layer, such as blemishes and uneven skin tone. At the same time, due to the edge-preserving properties of the joint filter itself, the clarity of key edges such as facial contours and light and shadow boundaries can still be maintained during the smoothing process, ultimately generating an optimized low-frequency base layer that is both smooth and retains structural information.

[0096] The optimized low-frequency base layer and the high-frequency detail layer are linearly superimposed and recombined to output a skin-smoothing and reshaped image.

[0097] Specifically, to simultaneously restore the skin's realistic texture and smooth skin tone in the final image, this step resynthesizes the optimized low-frequency base layer with the original high-frequency detail layer, which has not undergone any smoothing. In a preferred embodiment, this layer-by-layer recombination can be performed using linear overlay. This process ensures that the final output skin-reshaping image has both a uniform and delicate skin base and retains realistic texture details, effectively avoiding the "wax figure" effect caused by excessive blurring in traditional skin-reshaping techniques, and enhancing the naturalness of the skin texture. Figure 4 The diagram shown is a schematic of the surface texture optimization based on frequency layering of the present invention. It illustrates the process of decomposing an image into a low-frequency base layer and a high-frequency detail layer, selectively smoothing only the base layer, and finally recombining it with the high-frequency layer to preserve the texture with high fidelity.

[0098] Optionally, the optimized low-frequency base layer includes a smooth color component and an edge-preserving contour component, wherein:

[0099] The smooth color component is used to present basic skin tone information that has undergone blemish suppression and color equalization;

[0100] Specifically, this component represents the main effect of selective smoothing. In a preferred embodiment, when a bilateral filter is used to process the low-frequency base layer, the filter assigns a higher smoothing weight to pixel areas with similar colors, thereby effectively suppressing and filtering out minor imperfections, noise, and uneven color patches in the skin area. Therefore, this smoothed color component is the base skin tone information layer obtained after processing, with smooth and even transitions in color and light.

[0101] The edge-preserving contour component is used to retain high-frequency edge information that defines the facial structure and light and shadow boundaries and is identified and protected during smoothing.

[0102] Specifically, this component embodies the edge-preserving characteristic of selective smoothing. When performing smoothing calculations on pixels, this process considers both the spatial distance and color differences between pixels. For areas where color differences exceed a threshold, little or no smoothing is applied. Therefore, this edge-preserving contour component represents the sharp details that are fully recognized and preserved by the filter during the smoothing process, defining facial contours, feature boundaries, and the boundaries between light and shadow. Preserving this component is crucial for ensuring a natural final skin-smoothing effect and avoiding excessive blurring that leads to unclear features.

[0103] Optionally, the output beautified video data includes:

[0104] Based on the metadata recorded in the enhanced frame object queue, the target resolution and frame rate parameters are set, and the video encoder is initialized;

[0105] Specifically, this step prepares for the final video generation. The processing flow reads and utilizes metadata from the original video recorded in the enhanced frame object queue. This metadata primarily includes the video's original resolution and frame rate. Based on this metadata, the video encoder is configured and initialized. In a preferred embodiment, H.264 can be used as the video encoding standard to ensure good compression efficiency and broad compatibility.

[0106] The skin-smoothing and reshaping image sequence is submitted frame by frame to the video encoder according to the timestamp order in the enhanced frame object queue to generate a pure video basic stream;

[0107] Specifically, this step is responsible for encoding a series of independent skin-smoothing and reshaping images, processed by all the preceding beautification steps, into a continuous video data stream. In a preferred embodiment, a framework such as OpenCV can be used to feed the skin-smoothing and reshaping image sequence frame by frame into the video encoder initialized in the previous step, strictly following the timestamp order recorded in the enhancement frame object queue. After processing, the encoder outputs a pure video basic stream that does not contain audio information and conforms to a preset encoding standard.

[0108] Using the timestamp of the audio data packet as a synchronization reference, the pure video basic stream and the audio data packet are multiplexed and encapsulated into a digital media container format to generate beautified video data and output to the video terminal.

[0109] Specifically, to ensure precise audio-visual synchronization in the final video, this step uses the timestamp of the audio data packet as the synchronization benchmark. A multimedia merging framework, such as MoviePy, is used to multiplex the pure video stream generated in the previous step with the original audio data packet. This operation interweaves the video and audio data and encapsulates them into a standard digital media container format, such as MP4. After encapsulation, the final beautified video data is generated, which can be directly output to various video terminals, such as for short video platform publishing, live video streaming, or local high-definition playback.

[0110] Based on the same inventive concept, this invention also provides a video skin-beautifying system that integrates multiple color spaces and filtering, such as... Figure 5 As shown, the system includes:

[0111] The video preprocessing module is used to receive and read the raw video stream in real time, buffer and separate the raw video stream, output video data packets and audio data packets, decode the video data packets frame by frame to obtain unstructured image frames, and align and encapsulate the unstructured image frames and the audio data packets according to the timestamp to generate a frame object queue.

[0112] The face detection module is used to detect face regions and evaluate region confidence for the frame object queue. When the region confidence is lower than a preset face threshold, robust compensation detection is automatically started. Facial feature points are calibrated within the face region to generate a structured landmark set, which is then injected into the frame object queue to form an enhanced frame object queue.

[0113] The contour reshaping module is used to segment the facial region into a rigid protection region and a non-rigid deformation region based on the structured landmark set, calculate a basic deformation field and a local correction field in the non-rigid deformation region, perform vector weighted synthesis on the basic deformation field and the local correction field to generate a composite deformation flow field, and resample the original pixel grid of the enhanced frame object queue according to the composite deformation flow field to output the contour reshaping image.

[0114] The light and color reshaping module is used to project the contour reshaping image in parallel onto an orthogonal color feature space, which includes a luminance chromaticity space and a perceptual uniformity space. In the luminance chromaticity space, hue remapping is performed on the luminance channel and a compensation transformation is applied to the chromaticity channel. In the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted and fused and inversely transformed back to the standard color space to output the light and color reshaping image.

[0115] The skin smoothing and reshaping module is used to decompose the light and color reshaping image into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the local facial region of the structured landmark set, a nonlinear spatial value domain joint filtering process is applied to filter out blemishes and maintain edge awareness to obtain an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output the skin smoothing and reshaping image.

[0116] The video reconstruction module is used to initialize the video encoder based on the metadata in the enhanced frame object queue, send the skin smoothing and reshaping image sequence frame by frame into the video encoder in time stamp order to generate a pure video basic stream, and encapsulate the pure video basic stream and the audio data packet using multiplexing technology based on the timestamp of the audio data packet to output beautified video data.

[0117] To verify the feasibility of this invention, it was applied to a real-time beautification service on a video social platform to solve problems such as stiff contours, distorted skin tone, and audio-visual asynchrony caused by traditional algorithms. This embodiment processes 1080p, 30fps videos uploaded by users to verify the effectiveness of the method.

[0118] The input video stream is parsed into a queue of frame objects. Then, hybrid detection is used to detect faces in the video frames, and after localization, a structured landmark set is identified and injected into the frame objects to form an enhanced frame object queue. In the contour reshaping stage, a smoothed weighted mask is generated based on the structured landmark set to protect facial features. Next, using the tip of the nose and chin as dual centers, the basic deformation field and local correction field are calculated separately and synthesized into a composite deformation flow field. Natural face slimming is achieved through inverse mapping resampling technology. In the skin tone optimization stage, the image is processed in parallel using YCrCb and LAB color spaces. Brightness and chromaticity components are adjusted to improve skin whiteness and naturalness. After weighted fusion, the results are then globally fine-tuned for saturation and brightness using a universal color space, ultimately outputting a light and color reshaped image. In the skin smoothing stage, an adaptive control parameter set is generated based on face scale and deformation intensity. Subsequently, the image is decomposed into a low-frequency base layer and a high-frequency detail layer. Bilateral filter parameters are dynamically set according to the aforementioned parameter set, and smoothing is performed only on the low-frequency base layer. Finally, the optimized base layer is linearly overlaid with the original detail layer to reconstruct the skin-smoothing and reshaped image. All processed image frames are encoded into H.264 video streams and multiplexed with the original audio stream according to timestamps, then packaged into the final beautified video in MP4 format.

[0119] Table 1. Contour Reshaping Effect Data Table

[0120]

[0121] Table 2 Light Color Optimization Data Table

[0122]

[0123] Table 3. Comparison of Texture Optimization and Final Result

[0124]

[0125] As can be seen from the data in Tables 1 to 3, the method of the present invention, while achieving efficient contour shrinkage and natural skin tone optimization, improves skin smoothness and detail retention compared to traditional methods, has a higher processing frame rate, and significantly reduces the user negative feedback rate, which comprehensively proves the technical advantages of the present invention.

[0126] It should be noted that the functional division and information interaction between the various modules described above are logical, but in terms of physical implementation, they can be integrated on the same software platform or deployed in a distributed manner. The connections between them represent data flow and control flow, aiming to collaboratively achieve the objectives of this invention. The above descriptions are merely exemplary embodiments of this invention and should not be construed as limiting the scope of protection of this invention.

Claims

1. A video skin-beautification method integrating multiple color spaces and filtering, characterized in that, The method includes: The system receives and reads the raw video stream in real time, buffers and separates the raw video stream, outputs video data packets and audio data packets, decodes the video data packets frame by frame to obtain unstructured image frames, aligns and encapsulates the unstructured image frames and the audio data packets according to the timestamp, and generates a frame object queue. For the frame object queue, the face region is detected and the region confidence is evaluated. When the region confidence is lower than the preset face threshold, robust compensation detection is automatically started. Within the face region, facial feature points are calibrated, a structured landmark set is generated, and injected into the frame object queue to form an enhanced frame object queue. Based on the structured landmark set, the facial region is segmented into rigid protection regions and non-rigid deformation regions. A basic deformation field and a local correction field are calculated in the non-rigid deformation regions. The basic deformation field and the local correction field are then vector-weighted and synthesized to generate a composite deformation flow field. Based on this composite deformation flow field, the original pixel grid of the enhanced frame object queue is reverse-mapped and image interpolated to output a contour reshaping image. The output contour reshaping image includes: segmenting the facial region into rigid protection regions and non-rigid deformation regions based on the topological relationships of the structured landmark set; applying Gaussian smoothing filtering to the non-rigid deformation regions to generate a smoothing weight mask; calculating the basic deformation field within the non-rigid deformation regions with the nose tip landmark as the geometric center and the local correction field with the chin center landmark as the geometric center; and calculating the non-rigid deformation field based on a radial contraction model using the nose tip coordinates calibrated in the structured landmark set as the geometric center. The displacement vector of each pixel within the deformation region is used, while the local correction field focuses on more refined and natural optimization of the chin region to prevent insufficient deformation or abruptness. Using the chin center coordinates calibrated by the structured landmark set as the geometric center, a radial contraction model is also employed, but with a set of independent intensity and transition parameters to enhance the symmetrical contraction effect of the chin region. A composite deformation flow field is generated by vector-weighted synthesis of the basic deformation field and the local correction field. Based on the composite deformation flow field, the original pixel mesh in the enhanced frame object is reverse-mapped and image interpolated to output a contour reshaping image. A closed polygonal region is constructed by connecting the key points of the facial contour, and the region is filled with valid region values. Simultaneously, the polygonal regions formed by the key points corresponding to the eyes, nose, and mouth are filled with invalid region values, thus segmenting the facial region into a rigid protection region that does not participate in geometric transformation and a non-rigid deformation region that does participate in geometric transformation. The contour-reconstructed image is projected in parallel onto an orthogonal color feature space, which includes a luminance-chrominance space and a perceptual uniformity space. In the luminance-chrominance space, hue remapping is performed on the luminance channel and a compensation transformation is applied to the chrominance channel. In the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted and fused and inversely transformed back to the standard color space to output a light and color reconstructed image. The light and color reconstructed image is decomposed into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the local facial region of the structured landmark set, a nonlinear spatial domain joint filtering process is applied to filter out imperfections and maintain edge awareness, resulting in an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output a skin smoothing and reconstructed image. The video encoder is initialized based on the metadata in the enhanced frame object queue. The skin smoothing and reshaping image sequence is sent to the video encoder frame by frame in time stamp order to generate a pure video basic stream. Based on the timestamp of the audio data packet, the pure video basic stream and the audio data packet are encapsulated using multiplexing technology to output beautified video data.

2. The video beautification method integrating multiple color spaces and filtering according to claim 1, characterized in that, The queue of generated frame objects includes: The original video stream is buffered and parsed in real time, deconstructing it into a video data packet sequence and an audio data packet sequence; The video data packet sequence is decoded frame by frame to extract pixel information and timestamps, forming an unstructured image frame sequence; Using the timestamp of the audio data packet sequence as a synchronization reference, the unstructured image frame sequence is time-aligned with the audio data packet sequence, and the alignment result is encapsulated into a metadata frame object; The metadata frame objects are organized in chronological order and stored in a first-in-first-out cache to construct a frame object queue.

3. The video beautification method integrating multiple color spaces and filtering according to claim 1, characterized in that, The queue for forming enhanced frame objects includes: Frame objects are extracted from the frame object queue, statistical features of local gradient distribution in the image are detected, a first face region candidate box is generated, and the region confidence score is evaluated. If the confidence score of the region does not reach the face threshold, robust compensation detection is automatically activated to perform secondary detection and generate a second face region candidate box. The first or second face region candidate box is formatted and high-density facial feature point calibration is performed to generate a structured landmark set. The structured landmark set is injected into the frame object as enhanced data, and all the frame objects are reorganized to form an enhanced frame object queue.

4. The video beautification method integrating multiple color spaces and filtering according to claim 3, characterized in that, The method further includes: Perform geometric size analysis on the first or second face region candidate box, calculate the facial image scale, and obtain facial scale parameters. The vector amplitude statistical analysis of the composite deformation flow field was performed to quantify the profile reshaping intensity and obtain the deformation intensity index. The facial scale parameters and the deformation intensity index are nonlinearly fused to generate an adaptive control parameter set.

5. The video beautification method integrating multiple color spaces and filtering according to claim 1, characterized in that, The output light and color reconstructed image includes: Within the luminance-chrominance space, the luminance channel enhances dark areas and suppresses highlights, while the chrominance channel reduces the red component and increases the yellow component to generate first optimized color data. Within the perceived uniform space, the gain of the luminance channel is increased, while the red component of the chroma channel is reduced and the yellow component is increased to generate second optimized color data. The first optimized color data and the second optimized color data are weighted and synthesized to generate preliminary fused color data; The preliminary fused color data is transformed to a universal color space, and the saturation and brightness channels are globally fine-tuned to balance the overall tone, resulting in globally fine-tuned color data. The globally fine-tuned color data is inversely transformed back to the standard color space to output a light and color reconstructed image.

6. The video beautification method integrating multiple color spaces and filtering according to claim 4, characterized in that, The output skin smoothing and reshaping image includes: Based on the light and color reconstructed image, the basic light and shadow and color information are separated to obtain a low-frequency basic layer, and the surface texture and minute details are separated to obtain a high-frequency detail layer. Based on the adaptive control parameter set, the nonlinear spatial kernel size and color similarity weight are dynamically set to obtain a joint filter; Based on the joint filter, the low-frequency base layer is selectively smoothed to filter out imperfections and maintain edge awareness, thereby generating an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are linearly superimposed and recombined to output a skin-smoothing and reshaped image.

7. The video beautification method integrating multiple color spaces and filtering according to claim 6, characterized in that, The optimized low-frequency base layer includes a smooth color component and an edge-preserving contour component, wherein: The smooth color component is used to present basic skin tone information that has undergone blemish suppression and color equalization; The edge-preserving contour component is used to retain high-frequency edge information that defines the facial structure and light and shadow boundaries and is identified and protected during smoothing.

8. The video beautification method integrating multiple color spaces and filtering according to claim 1, characterized in that, The output beautified video data includes: Based on the metadata recorded in the enhanced frame object queue, the target resolution and frame rate parameters are set, and the video encoder is initialized; The skin-smoothing and reshaping image sequence is submitted frame by frame to the video encoder according to the timestamp order in the enhanced frame object queue to generate a pure video basic stream; Using the timestamp of the audio data packet as a synchronization reference, the pure video basic stream and the audio data packet are multiplexed and encapsulated into a digital media container format to generate beautified video data and output to the video terminal.

9. A video beautification system integrating multiple color spaces and filtering, applied to the video beautification method integrating multiple color spaces and filtering as described in any one of claims 1-8, characterized in that, The system includes: The video preprocessing module is used to receive and read the raw video stream in real time, buffer and separate the raw video stream, output video data packets and audio data packets, decode the video data packets frame by frame to obtain unstructured image frames, and align and encapsulate the unstructured image frames and the audio data packets according to the timestamp to generate a frame object queue. The face detection module is used to detect face regions and evaluate region confidence for the frame object queue. When the region confidence is lower than a preset face threshold, robust compensation detection is automatically started. Facial feature points are calibrated within the face region to generate a structured landmark set, which is then injected into the frame object queue to form an enhanced frame object queue. The contour reshaping module is used to segment the facial region into a rigid protection region and a non-rigid deformation region based on the structured landmark set. It calculates a basic deformation field and a local correction field in the non-rigid deformation region, performs vector weighted synthesis on the basic deformation field and the local correction field to generate a composite deformation flow field, and performs reverse mapping and image interpolation on the original pixel grid of the enhanced frame object queue based on the composite deformation flow field to output a contour reshaping image. A color reshaping module is used to project the contour reshaping image in parallel onto an orthogonal color feature space, which includes a luminance / chrominance space and a perceptual uniformity space. In the luminance / chrominance space, hue remapping is performed on the luminance channel, and a compensation transformation is applied to the chrominance channel. In the perceptual uniformity space, the three-dimensional channel coordinates are nonlinearly remapped to the target color coordinates. The output of the orthogonal color feature space is weighted and fused, and then inversely transformed back to the standard color space to output the color reshaping image. The output contour reshaping image includes: based on the topological relationship of the structured landmark set, segmenting the facial region into rigid protection regions and non-rigid deformation regions, and applying Gaussian smoothing filtering to the non-rigid deformation regions to generate a smoothing weight mask; based on the smoothing weight mask, within the non-rigid deformation regions, calculating the basic deformation field with the nose tip landmark as the geometric center, and calculating the local correction field with the chin center landmark as the geometric center; using the nose tip coordinates calibrated in the structured landmark set as the geometric center, and based on a... A radial contraction model is used to calculate the displacement vector of each pixel within the non-rigid deformation region. The local correction field focuses on a more refined and natural optimization of the chin region to prevent insufficient deformation or abruptness. Using the chin center coordinates calibrated by a structured landmark set as the geometric center, a radial contraction model is also employed, but with a set of independent intensity and transition parameters to enhance the symmetrical contraction effect of the chin region. A composite deformation flow field is generated by vector-weighted synthesis of the basic deformation field and the local correction field. Based on this composite deformation flow field, the original pixel mesh in the enhanced frame object is reverse-mapped and image interpolated to output a contour-reconstructed image. A closed polygonal region is constructed by connecting the key points of the facial contour, and the region is filled with valid region values. Simultaneously, the polygonal regions formed by the key points corresponding to the eyes, nose, and mouth are filled with invalid region values, thus segmenting the facial region into a rigid protection region that does not participate in geometric transformation and a non-rigid deformation region that does participate in geometric transformation. The skin smoothing and reshaping module is used to decompose the light and color reshaping image into a low-frequency base layer and a high-frequency detail layer. For the low-frequency base layer, combined with the local facial region of the structured landmark set, a nonlinear spatial value domain joint filtering process is applied to filter out blemishes and maintain edge awareness to obtain an optimized low-frequency base layer. The optimized low-frequency base layer and the high-frequency detail layer are then subjected to layered weighted fusion and reconstruction to output the skin smoothing and reshaping image. The video reconstruction module is used to initialize the video encoder based on the metadata in the enhanced frame object queue, send the skin smoothing and reshaping image sequence frame by frame into the video encoder in time stamp order to generate a pure video basic stream, and encapsulate the pure video basic stream and the audio data packet using multiplexing technology based on the timestamp of the audio data packet to output beautified video data.

Images