Video processing method and device, electronic equipment and storage medium
By dynamically switching facial landmark detection modes and adaptively adjusting makeup intensity, the detection accuracy and power consumption issues of mobile devices in scenarios with rapid facial movement are resolved, thereby improving the accuracy of facial detection and the makeup effect.
Patent Information
- Application Number
- CN202211220780.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-08
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-10-08
AI Technical Summary
Existing technologies struggle to balance power consumption and detection accuracy in fast-moving face scenarios on mobile devices, resulting in inaccurate face tracking boxes and impacting the effectiveness of smart beauty makeup.
By updating the key point displacement sequence based on the location information of facial key points, dynamically switching the processing mode, using interval detection or frame-by-frame detection, and combining sharpness adaptive makeup intensity adjustment, the detection of facial key points is optimized.
While reducing device power consumption, it improves the accuracy of facial key point detection and intelligent makeup effects, avoiding blurring and unnatural makeup when the face moves quickly.
Smart Images

Figure CN116092147B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a video processing method, apparatus, electronic device, storage medium, and program product. Background Technology
[0002] Facial landmark detection predicts the coordinates of key facial features without any prior knowledge of scale or location. In recent years, with the development of deep learning, facial landmark detection has become increasingly accurate and efficient. However, as application scenarios become more complex, more stringent requirements are being placed on facial landmark detection tasks in different scenarios and modes. For example, when users shoot personal videos or conduct live streams, intelligent makeup is almost an indispensable function. As a technology based on facial landmarks, intelligent makeup mainly uses the coordinates of facial landmarks to intelligently apply makeup to faces in videos or images, including adding lipstick, eyeshadow, eyeliner, colored contact lenses, etc. Therefore, the final makeup effect largely depends on the accuracy of the facial landmark detection results.
[0003] However, when faces move rapidly in a video, the current algorithm, even with tracking logic, suffers from inaccurate bounding boxes, especially when faces move quickly. Improving the accuracy of the bounding boxes requires frame-by-frame detection in the video, necessitating higher computing power from the mobile device. Therefore, existing methods struggle to balance power consumption on mobile devices with face detection accuracy in scenarios involving rapid face movement. Summary of the Invention
[0004] This disclosure provides a video processing method, apparatus, electronic device, storage medium, and program product to at least solve the problem in related technologies of balancing power consumption on mobile devices and face detection accuracy in scenarios with rapid face movement. The technical solution of this disclosure is as follows:
[0005] According to a first aspect of the present disclosure, a video processing method is provided, comprising:
[0006] Based on the facial key point location information of the current detection frame in the video to be processed, the key point displacement sequence is updated; the key point displacement sequence includes the key point displacement between each pair of adjacent detection frames in a series of consecutive detection frames before the current detection frame, and the key point displacement represents the degree of change in the facial key point position in each pair of adjacent detection frames.
[0007] Based on the updated keypoint displacement sequence, a target processing mode is determined; the target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and to perform facial keypoint detection processing on the next detection frame to obtain the facial keypoint position information of the next detection frame.
[0008] The target processing mode is either a first processing mode or a second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames.
[0009] In an exemplary embodiment, determining the target processing mode based on the updated keypoint displacement sequence includes:
[0010] If the displacement of each key point in the updated key point displacement sequence is less than the threshold, then the target processing mode is determined to be the first processing mode.
[0011] If any keypoint displacement in the updated keypoint displacement sequence is greater than or equal to the threshold, then the target processing mode is determined to be the second processing mode.
[0012] In one exemplary embodiment, after determining the target processing mode based on the updated keypoint displacement sequence, the method further includes:
[0013] If the target processing mode is the first processing mode, then the video frame located after the current detection frame and separated from the current detection frame by a preset number of frames is selected as the next detection frame;
[0014] The next detection frame is processed by facial landmark detection using the first processing mode to obtain the facial landmark location information of the next detection frame, including:
[0015] Perform face detection processing on the next detection frame to obtain the face bounding box of the next detection frame;
[0016] The next detection frame is cropped based on the face bounding box to obtain a face image;
[0017] The face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
[0018] In one exemplary embodiment, after determining the target processing mode based on the updated keypoint displacement sequence, the method further includes:
[0019] If the target processing mode is the second processing mode, then the video frame located after the current detection frame and adjacent to the current detection frame is selected as the next detection frame;
[0020] The next detection frame is processed by the second processing mode to detect facial key points, thereby obtaining the location information of the facial key points in the next detection frame, including:
[0021] Perform face detection processing on the next detection frame to obtain the face bounding box of the next detection frame;
[0022] The next detection frame is cropped based on the face bounding box to obtain a face image;
[0023] The face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
[0024] In an exemplary embodiment, the step of performing facial landmark detection processing on the face image to obtain the facial landmark location information of the next detection frame includes:
[0025] The face image is processed by facial landmark detection to obtain the initial facial landmark location information of the next detection frame;
[0026] Based on the initial facial key point location information, a new facial bounding box is obtained;
[0027] The new face bounding box is used to crop the next detection frame to obtain a new face image;
[0028] The new face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
[0029] In one exemplary embodiment, the method further includes:
[0030] For two consecutive detection frames selected from the video to be processed, facial clarity information is obtained based on the displacement of key points in the two consecutive detection frames.
[0031] Based on the positive correlation between facial clarity information and makeup intensity, makeup processing is applied to the faces in the two consecutive detection frames.
[0032] In an exemplary embodiment, obtaining facial clarity information based on the displacement of key points in the two consecutive detection frames includes:
[0033] The facial clarity information is obtained by processing the displacement of key points in two consecutive detection frames using a facial clarity measurement function.
[0034] The facial sharpness measurement function is determined based on the negative correlation between facial sharpness information and key point displacement.
[0035] In an exemplary embodiment, updating the key point displacement sequence based on the detected facial key point location information includes:
[0036] Obtain the facial key point location information of the previous detection frame of the current detection frame;
[0037] Based on the facial key point location information of the current detection frame and the facial key point location information of the previous detection frame, a new key point displacement is obtained;
[0038] The new keypoint displacement is added to the keypoint displacement sequence to obtain the updated keypoint displacement sequence.
[0039] According to a second aspect of the present disclosure, a video processing apparatus is provided, comprising:
[0040] The update unit is configured to update the key point displacement sequence based on the facial key point position information of the current detection frame in the video to be processed; the key point displacement sequence includes the key point displacement between each pair of adjacent detection frames in a series of consecutive detection frames before the current detection frame, and the key point displacement represents the degree of change in the facial key point position in each pair of adjacent detection frames.
[0041] The determining unit is configured to perform a target processing mode based on the updated key point displacement sequence; the target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and perform facial key point detection processing on the next detection frame to obtain the facial key point position information of the next detection frame.
[0042] The target processing mode is either a first processing mode or a second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames.
[0043] In an exemplary embodiment, the determining unit is further configured to perform the following: if all keypoint displacements in the updated keypoint displacement sequence are less than a threshold, then determine the target processing mode as the first processing mode; if any keypoint displacement in the updated keypoint displacement sequence is greater than or equal to the threshold, then determine the target processing mode as the second processing mode.
[0044] In an exemplary embodiment, the apparatus further includes a detection unit configured to perform the following actions if the target processing mode is a first processing mode: selecting a video frame located after the current detection frame and separated from the current detection frame by a preset number of frames as the next detection frame; performing face detection processing on the next detection frame to obtain a face bounding box of the next detection frame; performing cropping processing on the next detection frame based on the face bounding box to obtain a face image; and performing face key point detection processing on the face image to obtain face key point location information of the next detection frame.
[0045] In an exemplary embodiment, the detection unit is further configured to perform the following steps if the target processing mode is the second processing mode: selecting a video frame located after the current detection frame and adjacent to the current detection frame as the next detection frame; performing face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; performing cropping processing on the next detection frame based on the face bounding box to obtain a face image; and performing face key point detection processing on the face image to obtain the face key point location information of the next detection frame.
[0046] In an exemplary embodiment, the detection unit is further configured to perform facial landmark detection processing on the face image to obtain initial facial landmark location information of the next detection frame; obtain a new face bounding box based on the initial facial landmark location information; crop the next detection frame using the new face bounding box to obtain a new face image; and perform facial landmark detection processing on the new face image to obtain facial landmark location information of the next detection frame.
[0047] In one exemplary embodiment, the apparatus further includes a makeup unit configured to perform makeup processing on the face in the two consecutive detection frames selected from the video to be processed, based on the displacement of key points in the two consecutive detection frames; and to perform makeup processing on the face in the two consecutive detection frames according to the positive correlation between the face clarity information and the makeup intensity.
[0048] In an exemplary embodiment, the makeup unit is further configured to process the key point displacement in the two consecutive detection frames using a face clarity metric function to obtain the face clarity information; the face clarity metric function is determined according to a negative correlation between the face clarity information and the key point displacement.
[0049] In an exemplary embodiment, the updating unit is further configured to perform the following steps: obtaining facial key point location information of the previous detection frame of the current detection frame; obtaining a new key point displacement based on the facial key point location information of the current detection frame and the facial key point location information of the previous detection frame; and adding the new key point displacement to the key point displacement sequence to obtain an updated key point displacement sequence.
[0050] According to a third aspect of the present disclosure, an electronic device is provided, comprising:
[0051] processor;
[0052] Memory used to store the processor's executable instructions;
[0053] The processor is configured to execute the instructions to implement the method as described in any of the preceding methods.
[0054] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in any of the preceding claims.
[0055] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the method as described in any of the preceding claims.
[0056] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:
[0057] Based on the facial key point location information of the current detection frame in the video to be processed, the key point displacement sequence is updated. The key point displacement represents the degree of change in the position of the facial key points. The facial clarity information can be obtained from the degree of change in the position of the facial key points. Therefore, based on the updated key point displacement sequence, it can be accurately determined whether to perform facial key point detection at intervals or frame by frame in subsequent video frames. This method of dynamic mode switching based on the updated key point displacement sequence can reduce the power consumption of the device while ensuring the accuracy of face detection.
[0058] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0060] Figure 1 This is a flowchart illustrating a video processing method according to an exemplary embodiment.
[0061] Figure 2 This is a schematic diagram illustrating a dynamic mode switching strategy according to an exemplary embodiment.
[0062] Figure 3 This is a schematic diagram illustrating a process for facial landmark detection through a first processing mode, according to an exemplary embodiment.
[0063] Figure 4 This is a schematic diagram illustrating a process for facial landmark detection using a second processing mode, according to an exemplary embodiment.
[0064] Figure 5 It is a function image of the face sharpness measurement function F according to an exemplary embodiment.
[0065] Figure 6 This is a flowchart illustrating a video processing method according to another exemplary embodiment.
[0066] Figure 7 This is a schematic diagram of the overall framework of a video processing method according to an exemplary embodiment.
[0067] Figure 8 This is a structural block diagram of a video processing apparatus according to an exemplary embodiment.
[0068] Figure 9 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation
[0069] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0070] It should be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0071] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.
[0072] In one exemplary embodiment, such as Figure 1 As shown, a video processing method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:
[0073] In step S110, the key point displacement sequence is updated based on the facial key point position information of the current detection frame in the video to be processed. The key point displacement sequence includes the key point displacement between each pair of adjacent detection frames in a series of consecutive detection frames before the current detection frame. The key point displacement represents the degree of change in the position of facial key points in each pair of adjacent detection frames.
[0074] Among them, the displacement of key points can be used to determine the sharpness of faces in consecutive detection frames.
[0075] The number of keypoint displacements included in the keypoint displacement sequence can be a fixed value, denoted as M. That is, the keypoint displacement sequence can be used to store the keypoint displacements between pairs of adjacent detection frames in the most recently detected M+1 consecutive detection frames.
[0076] The location information of facial key points in the current detection frame can be obtained through either the first processing mode or the second processing mode.
[0077] The key points used to calculate the displacement of key points should be relatively fixed key points on the face that are not easily changed with facial expressions and movements, such as the tip of the nose and ears. Key points such as the mouth are not suitable for calculating the displacement of key points because they change more with speaking.
[0078] In the specific implementation, after completing the keypoint detection processing of a detection frame and obtaining the facial keypoint location information, a dynamic mode switching strategy can be determined to match the clarity of the face, specifying which processing mode to use for selecting and processing the next detection frame. Specifically, the keypoint displacement sequence can be updated based on the obtained facial keypoint location information to ensure that the keypoint displacements stored in the updated sequence are the most recent keypoint displacements, thus facilitating the determination of the target processing mode through the keypoint displacement sequence.
[0079] More specifically, updating the keypoint displacement sequence based on the obtained facial keypoint location information may include: obtaining new keypoint displacements based on the facial keypoint location information of the current detection frame and the facial keypoint location information of the previous detection frame, adding the new keypoint displacements to the keypoint displacement sequence, and removing the keypoint displacements that were stored earliest in the keypoint displacement sequence to obtain the updated keypoint displacement sequence.
[0080] In step S120, a target processing mode is determined based on the updated key point displacement sequence. The target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and to perform face key point detection processing on the next detection frame to obtain the face key point position information of the next detection frame.
[0081] The target processing mode is either the first processing mode or the second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames.
[0082] The next detection frame may be adjacent to the current detection frame, or it may be separated from the current detection frame by a preset number of frames.
[0083] In practice, the dynamic mode switching strategy monitors the changes in the positions of facial key points across m consecutive frames and sets a threshold. When the displacement of each key point in the updated key point displacement sequence is less than the threshold, it indicates that the change in face position is small, suggesting that the face is moving slowly and has high clarity. To reduce the power consumption and processing time of mobile devices, an interval detection method can be used to detect facial key points in the video to be processed, thus determining the target processing mode as the first processing mode. This ensures the temporal stability of facial key points in the detected frames while maintaining low power consumption on mobile devices.
[0084] When the displacement of any keypoint in the updated keypoint displacement sequence is greater than or equal to the threshold, it indicates that the face position has changed significantly. This suggests that the face is moving too fast and has low clarity. To avoid lag or false detection of the detected face region, a frame-by-frame detection method can be used to perform face keypoint detection processing on the video to be processed. That is, the target processing mode is determined to be the second processing mode. At this time, face detection by frame-by-frame can obtain more accurate face keypoints and ensure the accuracy of keypoint positions when the face moves quickly.
[0085] refer to Figure 2 This is a schematic diagram illustrating a dynamic mode switching strategy in an exemplary embodiment. Taking the tip of the nose as a key point for calculating the displacement, after detecting facial key points in each detection frame of the video to be processed, a dynamic mode switching strategy judgment can be performed. If the displacement of the nose tip key point is less than the threshold T1 in m consecutive frames, it is determined that there is no rapid facial movement, and the processing mode can be switched to the first processing mode or the first processing mode can continue to be used. If the displacement of the nose tip key point is greater than or equal to the threshold T1 in m consecutive frames, it is determined that there is rapid facial movement, and the mode is switched to the second processing mode or the second processing mode can continue to be used.
[0086] In the aforementioned video processing method, the keypoint displacement sequence is updated based on the facial keypoint position information of the current detection frame in the video to be processed. The keypoint displacement can represent the degree of change in the position of the facial keypoint, and the facial clarity information can be obtained from the degree of change in the position of the facial keypoint. Therefore, based on the updated keypoint displacement sequence, it can be accurately determined whether to perform facial keypoint detection at intervals or frame-by-frame in subsequent video frames. This dynamic mode switching method based on the updated keypoint displacement sequence can reduce the power consumption of the device while maximizing the accuracy of face detection.
[0087] In an exemplary embodiment, in step S120, determining the target processing mode based on the updated key point displacement sequence includes:
[0088] Step S1201: If the displacement of each key point in the updated key point displacement sequence is less than the threshold, then the target processing mode is determined to be the first processing mode.
[0089] Step S1202: If the displacement of any key point in the updated key point displacement sequence is greater than or equal to the threshold, then the target processing mode is determined to be the second processing mode.
[0090] In practice, if the displacement of each keypoint in the updated keypoint displacement sequence is less than the threshold, it indicates that the change in face position is small, and the face movement speed is slow. In this case, to reduce the power consumption and processing time of the mobile device, an interval detection method can be used to perform face keypoint detection processing on the video to be processed, that is, the target processing mode is determined to be the first processing mode. This ensures the temporal stability of face keypoints in video frames under the premise of low power consumption of the mobile device. When the displacement of the most recent keypoints in the updated keypoint displacement sequence is greater than or equal to the threshold, it indicates that the change in face position is large, and the face movement speed is determined to be too fast. To avoid lag or false detection of the detected face region, a frame-by-frame detection method can be used to perform face keypoint detection processing on the video to be processed, that is, the target processing mode is determined to be the second processing mode. At this time, face detection by frame can obtain more accurate face keypoints, ensuring the accuracy of keypoint positions when the face moves quickly.
[0091] In this embodiment, a second processing mode is used during periods of rapid face movement. Frame-by-frame face detection yields more accurate facial landmarks, ensuring the accuracy of landmark positions during rapid face movement. A first processing mode is used when the relative position of the face is stable, ensuring the temporal stability of facial landmarks within video frames while maintaining low power consumption on the mobile device. This dynamic mode switching strategy addresses the problem that existing methods cannot simultaneously balance mobile device power consumption and face detection accuracy in scenarios with rapid face movement.
[0092] In an exemplary embodiment, after determining the target processing mode based on the updated key point displacement sequence in step S120, the method further includes: if the target processing mode is the first processing mode, then selecting a video frame located after the current detection frame and separated from the current detection frame by a preset number of frames as the next detection frame.
[0093] The first processing mode is used to perform facial landmark detection processing on the next detection frame to obtain the facial landmark location information of the next detection frame, including: performing face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; cropping the next detection frame based on the face bounding box to obtain the face image; and performing facial landmark detection processing on the face image to obtain the facial landmark location information of the next detection frame.
[0094] In specific implementation, when the processing mode is the first processing mode, the (n+1)th video frame after the current detection frame can be used as the next detection frame according to the preset interval frame number n.
[0095] refer to Figure 3 This is a flowchart illustrating the process of facial landmark detection using a first processing mode in an exemplary embodiment. In this application, the first processing mode is triggered when there is no rapid facial movement. In the first processing mode, the process of detecting facial landmarks on the selected next detection frame includes: inputting the selected next detection frame into the face detection model M1 to obtain the face bounding box of the next detection frame; completing the face bounding boxes of the video frames in the two detection frames; cropping the next detection frame using the face bounding boxes to obtain a local face image in the next detection frame; inputting this local face image into the facial landmark detection model M2 to obtain the facial landmark position information of the next detection frame; updating the landmark displacement sequence based on the facial landmark position information of the next detection frame; returning to step S120 to determine the dynamic mode switching strategy again.
[0096] Furthermore, in the first processing mode, the facial key point detection of video frames between two adjacent detection frames can be based on the facial bounding box of the earlier detection frame among the two adjacent detection frames, without the need to reselect the facial bounding box, thereby reducing the risk of key point jitter.
[0097] In this embodiment, the selection of detection frames through a first processing mode interval reduces the computing power requirements of the mobile device, resulting in lower power consumption and shorter processing time. Simultaneously, the first processing mode also ensures the stability of keypoint detection performance, significantly reducing the risk of keypoint jitter.
[0098] In an exemplary embodiment, after determining the target processing mode based on the updated keypoint displacement sequence in step S120, the method further includes: if the target processing mode is the second processing mode, then selecting a video frame located after the current detection frame and adjacent to the current detection frame as the next detection frame.
[0099] The second processing mode is used to perform facial landmark detection processing on the next detection frame to obtain the facial landmark location information of the next detection frame, including: performing face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; cropping the next detection frame based on the face bounding box to obtain the face image; and performing facial landmark detection processing on the face image to obtain the facial landmark location information of the next detection frame.
[0100] In specific implementation, when the processing mode is the second processing mode, a video frame located after the current detection frame and adjacent to the current detection frame can be selected as the next detection frame.
[0101] Furthermore, in an exemplary embodiment, performing facial landmark detection processing on the face image to obtain facial landmark location information for the next detection frame includes: performing facial landmark detection processing on the face image to obtain initial facial landmark location information for the next detection frame; obtaining a new face bounding box based on the initial facial landmark location information; cropping the next detection frame using the new face bounding box to obtain a new face image; and performing facial landmark detection processing on the new face image to obtain facial landmark location information for the next detection frame.
[0102] refer to Figure 4 This is a schematic diagram illustrating the process of facial landmark detection using a second processing mode in an exemplary embodiment. In this application, the second processing mode is triggered when there is rapid facial movement. In the second processing mode, the process of detecting facial landmarks on the selected next detection frame includes: inputting the selected next detection frame into a face detection model M1 to obtain the face bounding box of the next detection frame; cropping the next detection frame using the face bounding box to obtain a local face image in the next detection frame; inputting this local face image into a facial landmark detection model M2 to obtain the initial facial landmark position information of the next detection frame; and obtaining a new face bounding box based on the initial face landmark position information. The facial landmark detection result is corrected using the new face bounding box. Specifically, the next detection frame can be cropped again using the new face bounding box to obtain a new face image; the new face image is then input into the facial landmark detection model again to obtain the facial landmark position information of the next detection frame.
[0103] In the above embodiments, face detection is performed frame by frame using the second processing mode, which avoids the problem of poor facial landmark detection caused by inaccurate input face bounding boxes. Therefore, this processing mode can effectively alleviate the problem of facial landmark lag in scenarios with fast face movement.
[0104] Understandably, intelligent makeup, as a technology based on facial landmarks, primarily uses the coordinates of these landmarks to apply makeup to faces in videos or images, including adding lipstick, eyeshadow, eyeliner, and colored contact lenses. Therefore, the final makeup effect largely depends on the accuracy of the facial landmark detection results. However, when faces in videos move rapidly, the current algorithm, even with tracking logic, suffers from inaccurate tracking bounding boxes, especially when the face moves quickly. This lag in the face tracking bounding boxes directly leads to a lag in facial landmark detection results, which in turn causes a lag in the makeup effect. Furthermore, rapid face movement also results in abrupt makeup application intensity, severely impacting the user experience of the intelligent makeup function. Moreover, video frames during rapid face movement exhibit noticeable motion blur, but the makeup intensity does not match the degree of blur, leading to an unnatural appearance due to a blurred face but excessively high makeup intensity.
[0105] To address the issue of unnatural makeup effects caused by motion blur in video frames during rapid facial movement, this application proposes a method for adaptive makeup intensity based on sharpness, building upon the aforementioned strategy of detecting facial key points through dynamic mode switching.
[0106] In one exemplary embodiment, the method for adaptive makeup intensity based on clarity includes:
[0107] Step S210: For two consecutive detection frames selected from the video to be processed, obtain facial clarity information based on the displacement of facial key points in the two consecutive detection frames.
[0108] Step S220: According to the positive correlation between facial clarity information and makeup intensity, makeup processing is performed on the faces in two consecutive detection frames.
[0109] In practical implementation, when applying makeup to personal video works or live streams, for two consecutive detection frames dynamically selected from the video, the facial keypoint position information of these two detection frames is obtained using the aforementioned facial keypoint detection method. Based on the facial keypoint position information, the change in the same keypoint is calculated as the facial keypoint displacement in the two consecutive detection frames. Then, the facial sharpness measurement function F is used to process the keypoint displacement in the two consecutive detection frames to obtain facial sharpness information. Finally, according to the positive correlation between facial sharpness information and makeup intensity, makeup processing is applied to the faces in the two consecutive detection frames. That is, when the face moves slowly and the sharpness is high, the corresponding makeup intensity can be greater, making the makeup clear. Conversely, when the face moves quickly and the sharpness is low, especially causing blurring, the makeup intensity is adaptively reduced to make the makeup more natural.
[0110] Furthermore, in an exemplary embodiment, step S210, which obtains facial clarity information based on the displacement of facial key points in two consecutive detection frames, includes: processing the displacement of key points in two consecutive detection frames using a facial clarity metric function to obtain facial clarity information; the facial clarity metric function is determined according to the negative correlation between facial clarity information and key point displacement.
[0111] Specifically, the facial sharpness measurement function F can be expressed as:
[0112] F(x) = cos(π (max(0.3,min(0.8,x) )-0.3 ))
[0113] Where x represents the displacement of the key point, and F(x) represents the output face clarity s.
[0114] refer to Figure 5 Let F be the graph of the facial sharpness metric, where the horizontal axis represents the keypoint displacement and the vertical axis represents the facial sharpness s. Multiplying the facial sharpness s by the initial makeup intensity yields the final adaptive makeup intensity value. Furthermore, it can be understood that within a certain range of keypoint displacement, facial sharpness can be considered constant; for example, in... Figure 5 In the process, the key point displacement range can be set to 0-0.3 and 0.8-1, and the facial clarity remains unchanged. Therefore, the makeup intensity also remains unchanged within these two key point displacement ranges.
[0115] In the above embodiments, by measuring facial clarity, and based on the positive correlation between makeup intensity and clarity, the makeup intensity is adaptively adjusted to apply makeup to the face. This avoids the abrupt phenomenon of a blurry face but obvious makeup, thereby solving the problem of unnatural makeup effects caused by motion blur of video frames during rapid facial movement.
[0116] In an exemplary embodiment, step S110, updating the key point displacement sequence based on the detected facial key point location information, includes:
[0117] Step S1101: Obtain the facial key point location information of the previous detection frame of the current detection frame;
[0118] Step S1102: Based on the facial key point location information of the current detection frame and the facial key point location information of the previous detection frame, obtain the new key point displacement.
[0119] Step S1103: Add the new keypoint displacement to the keypoint displacement sequence to obtain the updated keypoint displacement sequence.
[0120] In practice, the difference between the detected facial keypoint position information in the current video frame and the facial keypoint position information in the previous video frame can be calculated to obtain a new keypoint displacement. The new keypoint displacement is added to the keypoint displacement sequence, and the earliest keypoint displacement stored in the keypoint displacement sequence is removed to obtain an updated keypoint displacement sequence.
[0121] In this embodiment, after obtaining new key point displacements using the facial key point location information of the current video frame, the new key point displacements are added to the key point displacement sequence, and the key point displacements stored earliest in the key point displacement sequence are removed. This ensures that the number of key point displacements in the updated key point displacement sequence remains unchanged, so that the key point displacements in the key point displacement sequence used for subsequent dynamic mode switching determination are the key point displacements closest to the current time, thus ensuring the accuracy of the subsequently determined target processing mode.
[0122] In another exemplary embodiment, such as Figure 6 The diagram shown is a flowchart illustrating another video processing method according to an exemplary embodiment. In this embodiment, the method includes the following steps:
[0123] Step S610: If the position information of the face key points in the current detection frame of the video to be processed is obtained, update the key point displacement sequence according to the detected face key point position information.
[0124] Step S620: If the displacement of each key point in the updated key point displacement sequence is less than the threshold, then the target processing mode is determined to be the first processing mode.
[0125] Step S621: Select a video frame that is located after the current detection frame and is separated from the current detection frame by a preset number of frames as the next detection frame;
[0126] Step S622: Perform face detection processing on the next detection frame using the face detection model to obtain the face bounding box of the next detection frame;
[0127] Step S623: Crop the next detection frame based on the face bounding box to obtain the face image;
[0128] Step S624: Input the face image into the face key point detection model to obtain the face key point location information of the next detection frame;
[0129] Step S630: If the displacement of any key point in the updated key point displacement sequence is greater than or equal to the threshold, then the target processing mode is determined to be the second processing mode.
[0130] Step S631: Select the video frame that is located after the current detection frame and adjacent to the current detection frame as the next detection frame;
[0131] Step S632: Perform face detection processing on the next detection frame using the face detection model to obtain the face bounding box of the next detection frame;
[0132] Step S633: Crop the next detection frame based on the face bounding box to obtain the face image;
[0133] Step S634: Input the face image into the face key point detection model to obtain the initial face key point location information of the next detection frame;
[0134] Step S635: Obtain a new face bounding box based on the initial face key point location information;
[0135] Step S636: The next detection frame is cropped again using the new face bounding box to obtain a new face image;
[0136] Step S637: Input the new face image into the face landmark detection model to obtain the face landmark location information of the next detection frame.
[0137] In one exemplary embodiment, to facilitate understanding of the embodiments of this disclosure by those skilled in the art, specific examples will be described below in conjunction with the accompanying drawings. References Figure 7 This is a schematic diagram of the overall framework of a video processing method in an application example, such as... Figure 7 As shown, the overall technical solution can be divided into two parts: a first processing mode and a second processing mode. The two parts are dynamically coordinated through a dynamic mode switching strategy, including the following main steps:
[0138] (1) The initial user video first selects a processing mode for face detection. For example, the initial processing mode is the first processing mode, and the detection frame is selected by an interval of n frames.
[0139] (2) Input the detection frame into the face detection model M1 to obtain the face bounding box.
[0140] (3) The detection frame is cropped by the face bounding box to obtain the face image.
[0141] (4) Input the face image into the face key point detection model M2 to obtain face key points. In the second processing mode, it is also necessary to obtain a new face bounding box based on the face key points, use the new face bounding box to crop the detection frame again to obtain a new face image, and input the new face image into the face key point detection model M2 again to obtain face key points.
[0142] (5) Calculate the displacement of each key point of the nose tip in consecutive m frames (corresponding to the displacement of each key point in the key point displacement sequence), and combine it with the threshold to determine whether to switch modes in subsequent frames. Specifically, if the displacement of each key point of the nose tip in consecutive m frames is less than the threshold T1, the first processing mode is determined to be used in the subsequent frames; otherwise, the second processing mode is used.
[0143] (6) Based on the displacement of key points in two consecutive frames, the face motion speed measurement function F is used to measure and obtain the face clarity information.
[0144] (7) Based on the positive correlation between facial clarity information and makeup intensity, the makeup effect of adaptive makeup intensity is obtained.
[0145] The video processing method proposed in this application has the following beneficial effects:
[0146] 1. This method balances power consumption on mobile devices with high accuracy in facial landmark detection during rapid face movement, effectively addressing the issues of high power consumption and low accuracy of key points in fast-moving face scenarios compared to traditional methods, thus improving the user experience of smart beauty apps. During periods of rapid face movement, a second processing mode is used, employing frame-by-frame face detection to obtain more accurate facial landmarks, ensuring the accuracy of landmark positions during rapid face movement. During periods of relatively stable face position, a first processing mode is used, ensuring the temporal stability of facial landmarks within video frames while maintaining low power consumption on mobile devices.
[0147] 2. The proposed adaptive makeup intensity method can adjust makeup application based on facial sharpness, conforming to the real-world shooting scenarios and resulting in a more natural intelligent makeup effect. This method measures facial sharpness to establish a positive correlation between makeup intensity and sharpness. Specifically, when movement is slow and sharpness is high, the makeup intensity is greater, resulting in a clearer makeup look; conversely, when movement is fast and sharpness is low (especially causing blurring), the makeup intensity is reduced, making the makeup more natural and avoiding the abrupt phenomenon of a blurred face with obvious makeup.
[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0149] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.
[0150] Based on the same inventive concept, this disclosure also provides a video processing apparatus for implementing the video processing method described above.
[0151] Figure 8 This is a structural block diagram of a video processing apparatus according to an exemplary embodiment. (Refer to...) Figure 8 The device includes: an updating unit 810 and a determining unit 820, wherein,
[0152] The update unit 810 is configured to update the key point displacement sequence based on the facial key point position information of the current detection frame in the video to be processed; the key point displacement sequence includes the key point displacement between each pair of adjacent detection frames in a series of consecutive detection frames before the current detection frame, and the key point displacement represents the degree of change in the position of the facial key points in each pair of adjacent detection frames.
[0153] The determining unit 820 is configured to perform a target processing mode based on the updated key point displacement sequence. The target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and to perform face key point detection processing on the next detection frame to obtain the face key point position information of the next detection frame.
[0154] The target processing mode is either the first processing mode or the second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames.
[0155] In an exemplary embodiment, the determining unit 820 is further configured to perform the following: if all keypoint displacements in the updated keypoint displacement sequence are less than a threshold, then determine the target processing mode as a first processing mode; if any keypoint displacement in the updated keypoint displacement sequence is greater than or equal to the threshold, then determine the target processing mode as a second processing mode.
[0156] In an exemplary embodiment, the apparatus further includes a detection unit configured to perform the following actions if the target processing mode is a first processing mode: selecting a video frame located after the current detection frame and separated from the current detection frame by a preset number of frames as the next detection frame; performing face detection processing on the next detection frame to obtain a face bounding box of the next detection frame; performing cropping processing on the next detection frame based on the face bounding box to obtain a face image; and performing face key point detection processing on the face image to obtain face key point location information of the next detection frame.
[0157] In an exemplary embodiment, the detection unit is further configured to perform the following actions if the target processing mode is the second processing mode: selecting a video frame located after and adjacent to the current detection frame as the next detection frame; performing face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; performing cropping processing on the next detection frame based on the face bounding box to obtain a face image; and performing face key point detection processing on the face image to obtain the face key point location information of the next detection frame.
[0158] In an exemplary embodiment, the detection unit is further configured to perform facial landmark detection processing on the face image to obtain initial facial landmark location information of the next detection frame; obtain a new face bounding box based on the initial facial landmark location information; crop the next detection frame using the new face bounding box to obtain a new face image; and perform facial landmark detection processing on the new face image to obtain facial landmark location information of the next detection frame.
[0159] In one exemplary embodiment, the apparatus further includes a makeup unit configured to perform makeup processing on the face in the two consecutive detection frames selected from the video to be processed, based on the displacement of key points in the two consecutive detection frames, to obtain facial clarity information; and to perform makeup processing on the face in the two consecutive detection frames according to the positive correlation between facial clarity information and makeup intensity.
[0160] In an exemplary embodiment, the makeup unit is further configured to process the displacement of key points in two consecutive detection frames using a face sharpness metric function to obtain face sharpness information; the face sharpness metric function is determined according to the negative correlation between face sharpness information and key point displacement.
[0161] In an exemplary embodiment, the updating unit 810 is further configured to perform the following: obtain the facial key point position information of the previous detection frame of the current detection frame; obtain a new key point displacement based on the facial key point position information of the current detection frame and the facial key point position information of the previous detection frame; and add the new key point displacement to the key point displacement sequence to obtain an updated key point displacement sequence.
[0162] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0163] Figure 9 This is a block diagram illustrating an electronic device 900 for implementing a video processing method according to an exemplary embodiment. For example, the electronic device 900 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0164] Reference Figure 9 The electronic device 900 may include one or more of the following components: processing component 902, memory 904, power supply component 906, multimedia component 908, audio component 910, input / output (I / O) interface 912, sensor component 914, and communication component 916.
[0165] Processing component 902 typically controls the overall operation of electronic device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.
[0166] Memory 904 is configured to store various types of data to support the operation of electronic device 900. Examples of such data include instructions for any application or method operating on electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, optical disk, or graphene storage.
[0167] Power supply component 906 provides power to various components of electronic device 900. Power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 900.
[0168] Multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0169] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when electronic device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.
[0170] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0171] Sensor assembly 914 includes one or more sensors for providing state assessments of various aspects of electronic device 900. For example, sensor assembly 914 can detect the on / off state of electronic device 900, the relative positioning of components such as the display and keypad of electronic device 900, changes in position of electronic device 900 or its components, the presence or absence of user contact with electronic device 900, orientation or acceleration / deceleration of device 900, and temperature changes of electronic device 900. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.
[0172] Communication component 916 is configured to facilitate wired or wireless communication between electronic device 900 and other devices. Electronic device 900 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0173] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0174] In one exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of an electronic device 900 to perform the above-described method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0175] In one exemplary embodiment, a computer program product is also provided, which includes instructions that can be executed by a processor 920 of an electronic device 900 to perform the above-described method.
[0176] It should be noted that the above-mentioned apparatus, electronic equipment, computer-readable storage medium, computer program product, etc., may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.
[0177] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.
[0178] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A video processing method, characterized in that, include: Based on the facial key point location information of the current detected frame in the video to be processed, update the key point displacement sequence; The keypoint displacement sequence includes the keypoint displacement between each pair of adjacent detection frames in a series of consecutive detection frames preceding the current detection frame. The keypoint displacement represents the degree of change in the position of the facial keypoints in the pairs of adjacent detection frames. The keypoint displacement is negatively correlated with the facial clarity corresponding to the series of consecutive detection frames. Based on the updated key point displacement sequence, the target processing mode is determined. The target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and to perform facial key point detection processing on the next detection frame to obtain the facial key point location information of the next detection frame. The target processing mode is either a first processing mode or a second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames. The method further includes: For two consecutive detection frames selected from the video to be processed, facial clarity information is obtained based on the displacement of key points in the two consecutive detection frames. Based on the positive correlation between facial clarity information and makeup intensity, makeup processing is applied to the faces in the two consecutive detection frames.
2. The method according to claim 1, characterized in that, The determination of the target processing mode based on the updated key point displacement sequence includes: If the displacement of each key point in the updated key point displacement sequence is less than the threshold, then the target processing mode is determined to be the first processing mode. If any keypoint displacement in the updated keypoint displacement sequence is greater than or equal to the threshold, then the target processing mode is determined to be the second processing mode.
3. The method according to claim 1, characterized in that, After determining the target processing mode based on the updated keypoint displacement sequence, the following steps are also included: If the target processing mode is the first processing mode, then the video frame located after the current detection frame and separated from the current detection frame by the preset number of frames is selected as the next detection frame; The next detection frame is processed by facial landmark detection using the first processing mode to obtain the facial landmark location information of the next detection frame, including: Perform face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; The next detection frame is cropped based on the face bounding box to obtain a face image; The face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
4. The method according to claim 1, characterized in that, After determining the target processing mode based on the updated keypoint displacement sequence, the following steps are also included: If the target processing mode is the second processing mode, then the video frame located after the current detection frame and adjacent to the current detection frame is selected as the next detection frame; The next detection frame is processed by the second processing mode to detect facial key points, thereby obtaining the location information of the facial key points in the next detection frame, including: Perform face detection processing on the next detection frame to obtain the face bounding box of the next detection frame; The next detection frame is cropped based on the face bounding box to obtain a face image; The face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
5. The method according to claim 4, characterized in that, The step of performing facial landmark detection processing on the face image to obtain the facial landmark location information of the next detection frame includes: The face image is processed by facial landmark detection to obtain the initial facial landmark location information of the next detection frame; Based on the initial facial key point location information, a new facial bounding box is obtained; The new face bounding box is used to crop the next detection frame to obtain a new face image; The new face image is processed by facial landmark detection to obtain the facial landmark location information of the next detection frame.
6. The method according to claim 1, characterized in that, The step of obtaining facial clarity information based on the displacement of key points in two consecutive detection frames includes: The facial clarity information is obtained by processing the displacement of key points in two consecutive detection frames using a facial clarity measurement function. The facial sharpness measurement function is determined based on the negative correlation between facial sharpness information and key point displacement.
7. The method according to claim 1, characterized in that, The step of updating the keypoint displacement sequence based on the facial keypoint location information of the current detected frame in the video to be processed includes: Obtain the facial key point location information of the previous detection frame of the current detection frame; Based on the facial key point location information of the current detection frame and the facial key point location information of the previous detection frame, a new key point displacement is obtained; The new keypoint displacement is added to the keypoint displacement sequence to obtain the updated keypoint displacement sequence.
8. A video processing apparatus, characterized in that, include: The update unit is configured to update the keypoint displacement sequence based on the face keypoint location information of the current detected frame in the video to be processed; The keypoint displacement sequence includes the keypoint displacement between each pair of adjacent detection frames in a series of consecutive detection frames preceding the current detection frame. The keypoint displacement represents the degree of change in the position of the facial keypoints in the pairs of adjacent detection frames. The keypoint displacement is negatively correlated with the facial clarity corresponding to the series of consecutive detection frames. The determination unit is configured to perform the determination of the target processing mode based on the updated key point displacement sequence; The target processing mode is used to select the next detection frame after the current detection frame from the video to be processed, and to perform facial key point detection processing on the next detection frame to obtain the facial key point location information of the next detection frame. The target processing mode is either a first processing mode or a second processing mode. The first processing mode means selecting detection frames at preset frame intervals and performing facial key point detection processing on the selected detection frames. The second processing mode means selecting detection frames frame by frame and performing facial key point detection processing on the selected detection frames. The device also includes a makeup unit configured to perform a process of obtaining facial clarity information based on the displacement of key points in two consecutive detection frames selected from the video to be processed. Based on the positive correlation between facial clarity information and makeup intensity, makeup processing is applied to the faces in the two consecutive detection frames.
9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video processing method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the video processing method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Face key point detection method, device and equipment and storage medium
CN112016371A
Intelligent transcoding method and device for high-definition monitoring video
CN112188212A
Image beautification processing method and device, storage medium and electronic equipment
CN113538274A
Face detection method and device, terminal equipment and computer readable storage medium
CN114973344A
