Video processing method and device, and electronic device

Abstract

This application relates to a video processing method, apparatus, and electronic device. The method includes: acquiring facial feature information of at least one preset object; during video recording, performing face detection on the captured video frames, and identifying face regions in the detected face regions that do not match the facial feature information of the preset object as non-preset object face regions; performing privacy protection processing on each non-preset object face region in the video, and caching the original face image data of the non-preset object; after video recording is completed, in response to a restoration command triggered by a user based on the authorization of the non-preset object's appearance in the video, using the cached original face image data, restoring the privacy-protected face regions in the authorized non-preset object's appearance segment in the video to their original face images. This application can meet the privacy protection needs of passersby when filming in public places, while enhancing the integrity of video content and the flexibility of video processing.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video processing method, apparatus and electronic device. Background Technology

[0002] With the increasing popularity of short videos and vlogs (Video Blogs, or video diaries), users inevitably capture images of passersby when filming in public places. To protect the privacy of these individuals, current video editing software typically offers face blurring or mosaic effects, allowing users to apply privacy protection to specific areas of the video in post-production.

[0003] However, existing video processing methods mostly use direct pixel-level blurring. Once the blurring is completed, the original human image information is permanently destroyed, making it impossible to restore the original image when it is necessary to restore the appearance of passersby. This affects the integrity of the video content and the flexibility of video processing. Summary of the Invention

[0004] In view of the above, this application provides a video processing method, apparatus, and electronic device to solve at least one problem existing in the background art.

[0005] In a first aspect, embodiments of this application provide a video processing method, the method comprising: Obtain facial feature information of at least one preset object; During video recording, face detection is performed on the captured video frames. Face regions that do not match the facial feature information of the preset object are identified as non-preset object face regions. Privacy protection processing is performed on the face regions of each non-preset object in the video, and the original face image data of the non-preset object is cached; After the video recording is completed, in response to the user's restoration command triggered based on the non-preset object's appearance authorization, the cached original face image data is used to restore the face area of ​​the authorized non-preset object in the video, which has been processed for privacy protection, to the original face image.

[0006] In conjunction with the first aspect, in an optional implementation, caching the original face image data of the non-preset object includes: Assign a unique identifier to each detected non-preset object; For each non-preset object, perform the following operations: Record the duration of the non-preset object in the current monitoring window. The current monitoring window starts at the moment when the non-preset object is first detected during the current occurrence process and ends at the moment when the non-preset object is not detected for the first time in a preset number of consecutive frames. The original facial image data and corresponding on-screen clips of the non-preset objects are cached in real time, and the cached data is bound to the unique identifier. When the current monitoring window ends, depending on whether the non-preset object meets the preset conditions, the original face image data of the non-preset object that has been cached and the corresponding on-screen segment are selectively retained for face restoration operation after the video shooting is completed.

[0007] In conjunction with the first aspect, in an optional implementation, the non-preset object satisfies any of the following preset conditions: The on-camera authorization instruction for the non-preset object has been obtained; No authorization confirmation information was obtained regarding whether the non-preset object is allowed to enter the camera, and the non-preset object has been present in the current monitoring window for a preset duration.

[0008] In conjunction with the first aspect, in an optional implementation, the privacy protection processing performed on the face regions of each non-preset object in the video includes: For each non-preset object in the video, a mosaic effect is applied to the face region of the non-preset object. The size of the mosaic block used in the mosaic effect is negatively correlated with the distance between the non-preset object and the video shooting device.

[0009] In conjunction with the first aspect, in an alternative implementation, the method further includes: During video recording, based on the voice information and / or behavioral information of the non-preset object, an AI model is used to analyze the non-preset object's authorization to appear on camera.

[0010] In conjunction with the first aspect, in an optional implementation, the step of, after video recording is completed, responding to a recovery command triggered by a user based on authorization for non-preset subjects to appear in the video, using cached original facial image data, to restore the privacy-protected facial regions within the authorized non-preset subject segments of the video to their original facial images, includes: After the video recording is completed, a recovery selection interface is displayed, in which non-preset objects whose original facial image data has been retained are displayed in thumbnail form; In response to the user's restoration command for an authorized target non-preset object on the restoration selection interface, the system retrieves the original face image data associated with the target non-preset object from the cache, and restores the face region of the target non-preset object in the video segment that has been processed for privacy protection to the original face image.

[0011] In conjunction with the first aspect, in an optional implementation, the recovery selection interface has a preview function; the method further includes: In response to the user's preview operation on the recovery selection interface for the target non-preset object, at least one on-screen segment of the target non-preset object appearing in the video is played.

[0012] In conjunction with the first aspect, in an alternative implementation, the method further includes: After generating the final video file based on the face restoration operation on authorized non-preset objects, a data deletion prompt is output for all original face image data in the cache; In response to the user's confirmation of the data deletion prompt, all cached original face image data are deleted; If no confirmation is received from the user within a preset time, all cached original face image data will be automatically deleted.

[0013] Secondly, embodiments of this application provide a video processing apparatus, the apparatus comprising: The information acquisition module is used to acquire facial feature information of at least one preset object; The face detection module is used to perform face detection on the captured video frames during video shooting, and to determine the face regions that do not match the face feature information of the preset object as non-preset object face regions. The privacy protection module is used to perform privacy protection processing on the facial regions of each non-preset object in the video; The data caching module is used to cache the original face image data of the non-preset object; The face restoration module is used to restore the face regions of authorized non-preset objects in the video footage that have been processed for privacy protection to the original face images after the video is recorded, in response to a restoration command triggered by the user based on the authorization of non-preset objects to appear in the video, using the cached original face image data.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, the processor being configured to invoke instructions to cause the electronic device to execute the video processing method as described in any of the first aspects.

[0015] Fourthly, embodiments of this application provide a storage medium having an executable program stored thereon, wherein the executable program, when executed by a processor, implements the video processing method as described in any of the first aspects.

[0016] This application provides a video processing method, apparatus, and electronic device. By acquiring the facial feature information of a preset object (e.g., a Vlog creator), the detected facial regions are distinguished into preset objects and non-preset objects (e.g., passersby) during video shooting. Privacy protection processing is performed only on the facial regions of non-preset objects, and their original image data is cached, thereby avoiding the situation where the creator's face is mistakenly processed due to indiscriminate blurring. After shooting, the protected facial regions are restored to their original images using the cached original data based on the passerby's authorization, achieving reversibility of privacy protection. This solves the defect of existing technologies where blurring permanently destroys the original data and cannot be restored afterward. It balances the integrity of video content and the flexibility of video processing while protecting the privacy of passersby, meeting the actual needs of personalized creation scenarios such as Vlogs. Attached Figure Description

[0017] Figure 1 This is one of the flowcharts illustrating the video processing method provided in the embodiments of this application; Figure 2 for Figure 1 The flowchart of caching data in step S103 is shown below; Figure 3 for Figure 1 The flowchart of step S104 is shown below; Figure 4 A second schematic flowchart illustrating the video processing method provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of the video processing apparatus provided in the embodiments of this application. Detailed Implementation

[0018] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.

[0019] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0020] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0021] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] Figure 1 This is a schematic flowchart illustrating a video processing method provided in an embodiment of this application. (See attached document.) Figure 1 As shown, the video processing method includes the following steps: S101: Obtain facial feature information of at least one preset object; S102: During video recording, face detection is performed on the captured video frames. Face regions in the detected face regions that do not match the face feature information of the preset object are identified as face regions of non-preset objects. S103: Perform privacy protection processing on the face region of each non-preset object in the video, and cache the original face image data of the non-preset object; S104: After the video recording is completed, in response to the user's restoration command triggered based on the non-preset object's on-camera authorization, the cached original face image data is used to restore the face area of ​​the authorized non-preset object in the video segment that has been processed for privacy protection to the original face image.

[0023] In this embodiment, the video processing method can be applied to terminal devices with video shooting and processing functions. Terminal devices may include, but are not limited to, smartphones, tablets, digital cameras, camcorders, and personal computers.

[0024] Preset subjects refer to individuals who need to be clearly presented in the video, such as the vlog creator, interview host, or participants who have given explicit consent to appear on camera throughout. The facial features of these subjects can be pre-acquired and stored in an identity whitelist, and their faces will not be subject to any privacy protection processing during filming.

[0025] Non-preset subjects refer to people other than the preset subjects who appear in the video shooting, such as passers-by or other people. Their faces will be processed in real time during the shooting process to protect their privacy.

[0026] In step S101 above, facial feature information of at least one preset object can be obtained before video recording is performed.

[0027] Facial feature information can be obtained in various ways. For example, before taking a picture, the target subject can be guided to enter their facial image within the viewfinder, or a photo containing the target subject's face can be selected from the album. After obtaining the target subject's facial image, the terminal device can use a facial recognition algorithm to extract facial features. For example, an AI feature extraction algorithm based on a convolutional neural network can extract the coordinates and feature vectors of key facial points to generate a facial feature template, thereby forming a whitelist of the target subject's face.

[0028] To further enhance data security, a symmetric encryption algorithm can be used to encrypt the facial feature template of the preset object, and then the encrypted template can be stored in the local secure storage area of ​​the terminal device.

[0029] In step S102 above, during video recording, the terminal device captures video frames in real time and performs face detection on each frame. For each detected face region in each frame, its facial features are extracted and compared with a whitelist of preset objects. If the features of a certain face region match the features of any preset object's face region in the whitelist, then the face region is determined to belong to the preset object, and no privacy processing is performed on it subsequently to ensure that the preset object's face is clearly visible in the video; if the features do not match, then the face region is determined to be a non-preset object's face region (e.g., a passerby). The feature comparison can use methods such as feature vector distance calculation to quickly and accurately distinguish between preset objects and non-preset objects.

[0030] In step S103 above, for each face region in the real-time captured video that is identified as a non-preset object, the terminal device performs privacy protection processing and caching of the original face image data. Privacy protection processing may include, for example, applying mosaic, Gaussian blur, or pixelation to the face regions of non-preset objects in the video. For face regions of preset objects, there is no need to perform privacy protection processing or caching of the original face image data.

[0031] Here, raw face image data refers to the unprocessed pixel data directly extracted from the face regions of non-preset objects detected by the terminal device after face detection is performed on the captured video frames during video recording. Any raw face image data of a non-preset object contains complete facial visual information of that non-preset object in the current frame.

[0032] It's worth noting that when caching data, the terminal device doesn't cache the entire video stream. Instead, it creates a separate dynamic buffer for each non-preset object. For each video frame featuring that non-preset object, the terminal device stores the raw pixel data of its face region along with the timestamp of that video frame into the corresponding buffer. As shooting progresses, a series of raw face image data arranged chronologically accumulates in the buffer. This method of extracting and storing raw data frame by frame ensures that facial images with completely consistent quality with the original image can be obtained during post-production restoration, thus achieving high-precision reconstruction.

[0033] In step S104 above, after the video recording is completed, the user can trigger a recovery command on the terminal device based on the actual authorization obtained during or after the recording (e.g., the passerby agrees to appear in the video on-site, or authorization is obtained afterward by clicking a link). When the terminal device receives a recovery command for a certain authorized non-preset object, it retrieves the previously cached original facial image data of the non-preset object, and replaces or restores the facial areas that have been processed for privacy protection in all on-screen segments of the object in the video frame by frame with the original facial images, thereby restoring the passerby's clear face.

[0034] Compared to methods that perform privacy protection processing after video recording, this application performs privacy protection and facial image data caching in real time during video recording. On the one hand, this real-time processing protects the privacy of passersby during the recording stage, reducing the possibility of privacy leaks. On the other hand, it simultaneously distinguishes between preset subjects (e.g., Vlog creators) and non-preset subjects (e.g., passersby) during recording, and protects the privacy of the facial regions of non-preset subjects. Users do not need to spend a lot of time in post-production marking or manually adjusting the blurred areas frame by frame; the video can be easily edited after it is generated (e.g., restoring the faces of authorized passersby), effectively improving creative efficiency. In addition, the real-time caching of the original facial image data of non-preset subjects provides a foundation for subsequent face restoration, allowing the protected areas to be restored to their original state after recording, based on the actual authorization situation. This reversible mechanism satisfies the privacy protection needs during the recording stage while preserving the integrity of the video content and the flexibility of post-processing.

[0035] In some embodiments, the above step S103, which performs privacy protection processing on the face region of each non-preset object in the video, may include: applying mosaic processing to the face region of each non-preset object in the video, wherein the size of the mosaic block used in the mosaic processing is negatively correlated with the distance between the non-preset object and the video shooting device.

[0036] Here, the size of the mosaic blocks used in the mosaic processing is negatively correlated with the distance between the non-preset object and the video shooting device, so that the larger the imaging size of the non-preset object in the picture, the larger the applied mosaic blocks are.

[0037] The distance between a non-preset object and the shooting device can be estimated in several ways. For example, the relative distance can be estimated using the size of the face detection bounding box; the larger the pixel area occupied by a face in the image, the closer the object is to the shooting device, and vice versa. In practice, the relative distance between the non-preset object and the shooting device can be calculated by combining the ratio of the preset face reference size to the actual detected face pixel width or height with the camera imaging parameters.

[0038] In this embodiment, when a non-preset object is close to the video recording device and occupies a large area in the frame, a larger-grained mosaic block is used for occlusion; when the object is far away and occupies a small area in the frame, a smaller-grained mosaic block is used to maintain the naturalness of the image while ensuring privacy. This adaptive processing method effectively alleviates the problem of insufficient occlusion in close-up shots and excessive occlusion in distant shots caused by fixed-grained mosaics, improving the overall viewing experience and user experience of the video.

[0039] To optimize storage resources while protecting privacy, in some embodiments, such as Figure 2 As shown, in step S103 above, caching the original face image data of non-preset objects may include: S201: Assign a unique identifier to each non-preset object detected; S202: For each non-preset object, execute steps S2021 to S2023; S2021: Record the duration of non-preset objects in the current monitoring window. The current monitoring window starts at the moment when a non-preset object is first detected during the current occurrence process and ends at the moment when the non-preset object is not detected for the first time in a preset number of consecutive frames. S2022: Real-time caching of the original facial image data and corresponding on-screen footage, while binding the cached data with a unique identifier; S2023: At the end of the current monitoring window, depending on whether the non-preset object meets the preset conditions, the original face image data of the non-preset object that has been cached and the corresponding on-screen segments are selectively retained for face restoration operation after the video shooting is completed.

[0040] In this embodiment, the terminal device assigns a unique identifier to each detected non-preset object, such as face A, face B, etc., so that each non-preset object can be independently tracked and managed subsequently. For each non-preset object with an assigned unique identifier, the terminal device records its appearance duration within the current monitoring window by executing step S2021, and synchronously caches the object's original face image data and its corresponding on-screen segment by executing step S2022. Simultaneously, the cached data is bound to the unique identifier to ensure that the original face image data and its corresponding on-screen segment of each non-preset object can be accurately located. It is understood that steps S2021 and S2022 can be executed simultaneously; that is, when the terminal device detects a non-preset object in each frame, it synchronously performs the cumulative update of its appearance duration within the current monitoring window and the caching operation of the original face image data and on-screen segment.

[0041] Here, a non-preset object may appear once or multiple times during the shooting process. The current appearance process refers to the complete out-of-frame segment of the non-preset object from the moment it enters the shooting frame to the moment it leaves the shooting frame. The current monitoring window starts at the moment when the non-preset object is first detected in the current appearance process and ends at the moment when the non-preset object is not detected for the first time in a preset number of consecutive frames. Within each monitoring window, the terminal device continuously monitors whether the object appears in the video frames. When the object stays in the frame continuously, its appearance duration dynamically increases over time; only when the object is not detected in a preset number of consecutive video frames (for example, if the video frame rate is 30 frames / second, it can be set to 90 consecutive frames, i.e., 3 seconds) does the terminal device confirm that it has left the frame, the current appearance process ends, and the current monitoring window for the non-preset object ends.

[0042] It should be noted that detection may be interrupted during shooting due to factors such as obstruction, head turning, or the object briefly leaving the frame. As long as the number of frames with such interruption does not reach the preset consecutive number, even if the object temporarily disappears from the frame, it is still considered a continuation of the same occurrence process, and the occurrence duration is paused and accumulated until the object reappears in the frame. Only when the number of consecutive undetected frames reaches the preset threshold is the occurrence process considered to have ended.

[0043] In step S2021, the terminal device records the appearance duration of each non-preset object within the current monitoring window. The appearance duration refers to the cumulative duration during which the non-preset object is actually detected from the start time of the current monitoring window to the current time. For example, the appearance duration can be obtained by counting the total number of frames detected during the current appearance of the non-preset object and combining it with the video frame rate.

[0044] In step S2023, for each non-preset object, when the current monitoring window of the non-preset object ends, the terminal device selectively retains its cached original face image data and corresponding on-screen segments based on whether the non-preset object meets the preset conditions, so that the face restoration operation of the non-preset object can be performed when authorization is obtained after shooting.

[0045] In this embodiment, when the current monitoring window for a non-preset object ends, the terminal device selectively retains the cached original facial image data of the non-preset object based on whether the non-preset object meets preset conditions. By selectively retaining cached data, the system can preserve pedestrian data with potential recovery value, optimizing storage efficiency while ensuring privacy and security.

[0046] In some embodiments, the non-preset object in step S2023 above meets the preset conditions including any one of the following: an authorization instruction for the non-preset object to enter the camera has been obtained; no authorization confirmation information on whether the non-preset object is allowed to enter the camera has been obtained, and the appearance duration of the non-preset object in the current monitoring window reaches a preset duration.

[0047] Understandably, in actual shooting scenarios, if a non-preset subject (such as a passerby) wishes to be filmed, their behavior typically involves continuous movement or lingering within the filming range, rather than deliberately avoiding or quickly leaving the camera. Correspondingly, if the duration of a single appearance by this non-preset subject easily reaches or exceeds the preset duration, the system will fully retain their cached original facial image data and corresponding on-camera segments. Furthermore, if the system receives an authorization instruction from the non-preset subject (e.g., a passerby explicitly informing the photographer of their consent to be filmed via gestures or voice), regardless of whether their appearance duration reaches the preset duration, the system will also mark their cached data as retained. After video recording is complete, for non-preset subjects who have obtained on-camera authorization, the retained data can be fully restored to on-camera segments based on user recovery actions; for non-preset subjects who have not obtained on-camera authorization, the retained data will be deleted after recording to ensure privacy compliance.

[0048] Here, the specific value of the preset duration mentioned above is not fixed and can be flexibly configured by the user according to the actual shooting scenario and privacy protection needs. For example, the terminal device can provide a settings interface, allowing users to adjust relevant parameters as needed before or during shooting. As an example, the preset duration can be set to a value between 5 and 30 seconds. For instance, in scenarios such as street photography or event recording, it can be set to 10 seconds; in interviews or fixed-scene shooting, it can be set to 20 seconds. This range can effectively filter out passersby who are only briefly passing by, while also covering non-preset subjects who intend to participate in the shooting.

[0049] In some embodiments, the non-preset object in step S2023 not meeting the preset conditions includes at least one of the following: an entry rejection indication for the non-preset object has been obtained; authorization confirmation information regarding whether the non-preset object is allowed to enter the camera has not been obtained; and the appearance duration of the non-preset object in the current monitoring window has not reached the preset duration.

[0050] In this embodiment, if the duration of a non-preset object's appearance in the current monitoring window does not reach the preset duration (for example, the preset duration is 20 seconds, but the non-preset object only appears for 5 seconds), and no entry confirmation information is obtained, the system can determine that the non-preset object is a brief accidental entry or an invalid appearance. Alternatively, if entry rejection information indicating that the non-preset object is not allowed to appear in the camera has been obtained, in order to save storage resources and protect the privacy of passersby, the system will clear the original face image data and its corresponding appearance segment that have been cached during the appearance of the object from the cache area.

[0051] In this way, terminal devices can differentiate the processing of facial data from non-preset individuals. For passersby whose cached data is deleted due to short appearances (before reaching a threshold) or who explicitly refuse to appear on camera, data clearing will not affect the integrity of the video content. For passersby who have obtained authorization to appear on camera, or those who have not obtained authorization confirmation but whose appearances reach a threshold for extended periods, retaining their cached data can provide data support for possible subsequent facial reconstruction. This reduces storage overhead and mitigates the risk of long-term retention of private data while ensuring recoverability.

[0052] In some examples, when the duration of any non-preset object in the current occurrence process reaches the preset duration, the terminal device marks its cached original face image data as reserved to ensure that its original face image data will not be automatically deleted.

[0053] In some embodiments, when any non-preset object is detected, it is determined whether the historical cached data of that non-preset object has been marked as reserved. If so, without obtaining authorization confirmation information regarding whether the non-preset object is allowed to appear on camera, the original facial image data corresponding to its current on-camera segment and the on-camera segment are cached in real time, and newly appearing on-camera segments are continuously added to the cache record corresponding to the non-preset object. That is, if the cached data of any non-preset object is marked as reserved, the subsequent original facial image data of that non-preset object can continue to be cached in real time without obtaining authorization confirmation information regarding whether the non-preset object is allowed to appear on camera, and is also treated as reserved data. In this way, after the video is recorded, it can be determined whether to restore all on-camera segments related to the reserved data based on whether the non-preset object's on-camera authorization has been obtained. If on-camera refusal information for a non-preset object is obtained, its current cached data and its already reserved historical cached data are deleted, and the binding relationship between the passerby's unique identifier and the cached data is released. This enables timely cleanup of non-preset object data that has been explicitly refused on camera, reducing the risk of privacy data retention.

[0054] For example, suppose a user is filming a vlog, and the terminal device has a preset duration threshold of 5 seconds. When a passerby first appears in the frame, the terminal device assigns a identifier "Face 01" to them and begins caching their original facial image data and corresponding on-screen segment in real time, while recording the duration of their appearance in this round. If the passerby stays in the frame for 3 seconds and then turns away, since the appearance duration is only 3 seconds, which does not reach the 5-second threshold, and the passerby is not detected for a preset number of consecutive frames, the terminal device automatically deletes all cached data for the passerby, freeing up storage space. If a passerby remains in the frame continuously, and when the timer reaches 5 seconds, if the passerby neither gives a refusal to appear in the frame nor a permission to appear in the frame, the terminal device will mark all data cached in the previous 5 seconds as reserved. If the passerby then remains in the frame for another 3 seconds before leaving, the data for these subsequent 3 seconds will also be cached in real time and automatically reserved. In this way, the passerby's total 8 seconds of on-screen footage will be completely preserved, so that the facial image can be restored based on the passerby's permission to appear in the frame after the video is recorded.

[0055] In some embodiments, the method further includes: during video recording, using an AI model to analyze the on-camera authorization status of the non-preset object based on its voice information and / or behavioral information.

[0056] For audio data, the terminal device performs speech detection, converts the speech into text, and matches it against a pre-defined keyword library. For example, it identifies words like "can film" or "agree" indicating authorization to appear on camera, or words like "don't film" or "delete" indicating refusal to appear on camera. The analysis results are recorded as the passerby's willingness to appear on camera. For video data, the terminal device analyzes the data using an action recognition model to identify whether the passerby performs actions related to their willingness to appear on camera. For example, nodding or waving to agree indicates authorization to appear on camera, while shaking their head, waving their hand, or blocking the camera indicates refusal to appear on camera. The action recognition model can employ a lightweight spatiotemporal convolutional network, ensuring both recognition accuracy and real-time processing requirements.

[0057] In some examples, to ensure the smoothness of the video shooting process, the AI ​​model can use compression, quantization and other technologies to achieve a lightweight design, and use the NPU of the terminal chip for hardware acceleration to ensure that real-time processing does not affect the shooting frame rate.

[0058] The analysis results can be stored as auxiliary information and unique identifiers for non-preset objects, and presented as icons, labels, or sorting methods in the recovery selection interface after shooting. For example, passersby who are identified as having consented to appear in the shot can be marked "Authorized to appear in the shot"; passersby who are identified as having a tendency to refuse can be marked "Refuse to appear in the shot", helping creators quickly determine whether to obtain permission for non-preset objects to appear in the shot and perform the corresponding recovery operation.

[0059] In some embodiments, such as Figure 3 As shown, step S104 above, after the video recording is completed, responds to the recovery command triggered by the user based on the authorization of non-preset objects to appear in the video, and uses the cached original face image data to restore the face region of the authorized non-preset object in the video segment that has been processed for privacy protection to the original face image, which may include: S301: After video recording is completed, the recovery selection interface is displayed. The recovery selection interface displays non-preset objects with the original face image data retained in the form of thumbnails. S302: In response to the user's restoration command for the authorized target non-preset object on the restoration selection interface, retrieve the original face image data associated with the target non-preset object from the cache, and restore the face area of ​​the target non-preset object in the video segment that has been processed for privacy protection to the original face image.

[0060] In step S301, after video recording is completed, a floating icon can be displayed on the recording preview screen to prompt the user to enter the recovery selection interface. After entering the recovery selection interface, the terminal device displays all non-preset objects whose original facial image data has been retained in the form of a list, grid, or card. Each preset object can be accompanied by information such as facial thumbnail, time interval of appearance, or frequency of appearance, to facilitate user identification and filtering.

[0061] In step S302, based on the authorization obtained during or after shooting, the authorized target non-preset object can be determined. Then, the user selects the target non-preset object whose face needs to be restored on the recovery selection interface by clicking, checking, or swiping. Responding to the user's recovery command, the terminal device retrieves the original face image data associated with the unique identifier of the target non-preset object from the cache, and replaces the privacy-protected face regions in the corresponding video segments frame by frame based on this data, ultimately restoring the non-preset object's face to its original state.

[0062] In some possible implementations, the recovery operation can be performed in batches, that is, the user selects multiple authorized non-preset objects at once and restores them all at once; or it can be performed one by one in real time, that is, the face recovery operation for each non-preset object is selected.

[0063] In this embodiment, through the above steps, users do not need to manually locate and mark the position of the non-preset object that needs face restoration in the video. They can complete the accurate restoration simply by selecting it through the interface, which effectively improves the efficiency and convenience of video processing.

[0064] In some embodiments, the recovery selection interface has a preview function; the method further includes: in response to the user's preview operation on the recovery interface for a target non-preset object, playing at least one on-screen segment of the target non-preset object appearing in the video.

[0065] There are several ways to implement the preview function. For example, when a user hovers the cursor over or long-presses a thumbnail of a non-preset object, a small window automatically pops up to play key clips of that non-preset object in the video; or the user double-clicks the thumbnail to enter full-screen preview mode and view a collection of all clips featuring that non-preset object. During the preview, a timeline marker can be displayed, indicating the non-preset object's position in the video, and users can drag the progress bar to quickly browse. Furthermore, the preview interface can provide interaction information between the non-preset object and preset objects (such as vlog creators), such as the duration of their on-screen interaction, whether there is dialogue or gesture communication, etc. This information can be obtained through analysis of audio and video data collected during filming, helping users determine whether to obtain permission for the non-preset object to appear on screen.

[0066] In this embodiment, by restoring the preview function of the selection interface, users can quickly review the interactions of non-preset objects whose cached data is retained in the video, thereby facilitating accurate judgment on whether to obtain authorization for the non-preset object to appear on camera. This reduces the likelihood of misjudging or missing objects whose faces need to be restored due to user memory lapses, thus improving the user experience.

[0067] In some embodiments, the method further includes: after generating a final video file based on the face recovery operation on an authorized non-preset object, outputting a data deletion prompt for all cached original face image data; deleting all cached original face image data in response to the user's confirmation operation of the data deletion prompt; and automatically deleting all cached original face image data if no confirmation operation is received from the user within a preset time.

[0068] After completing all recovery operations and generating the final video file, the terminal device will initiate the data deletion process. This deletion process can be triggered in one of the following ways: One way is for the terminal device to automatically pop up a dialog box asking the user whether to delete all original passerby data; upon user confirmation, deletion is executed immediately. Another way is to provide an automatic cache clearing switch in the application settings; when enabled by the user, the terminal device will silently delete relevant data after each video export and will display a notification in the notification bar indicating the clearing result. To prevent accidental deletion or forgetting by the user, a countdown automatic deletion mechanism can also be designed; for example, if there is no response within 30 seconds after a pop-up prompt, deletion will be executed automatically.

[0069] In this embodiment, the cached data of the original facial image is stored locally on the terminal and is deleted by the user upon confirmation or automatically after the final video is generated, thereby reducing the risk of privacy data leakage caused by the original facial data flowing through multiple stages or being stored for a long time.

[0070] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0071] In the various embodiments of the specification, some or all of the steps and their optional implementations can be arbitrarily combined with some or all of the steps in other embodiments, or arbitrarily combined with the optional implementations in other embodiments.

[0072] Next, combined Figure 4 The video processing method provided in the embodiments of this application will be further described.

[0073] like Figure 4 As shown, the method may include the following steps: S1: Preset facial information of the subject is entered and AI feature extraction is performed; S2: Generate a facial feature template and store it encrypted; S3: Start Vlog recording and capture video stream in real time; S4: Frame-by-frame face detection; S5: AI feature comparison and language behavior judgment; S6: Determine whether the face region of the current frame belongs to the preset object; if yes, proceed to step S7; otherwise, proceed to step S8.

[0074] S7: Preserve the original face region; proceed to step S11 after step S7; S8: Apply mosaic processing in real time; S9: Assign a unique face identifier and cache face data, while recording the duration of non-preset object faces in the current monitoring window; the current monitoring window starts at the moment when a non-preset object is first detected in the current occurrence process, and ends at the moment when the non-preset object is not detected for the first time in a consecutive preset number of frames. S10: For non-preset objects that have obtained an on-camera authorization instruction, or for non-preset object faces whose duration has reached the threshold, retain the cached face data and the corresponding on-camera segments; for non-preset objects that have obtained an on-camera rejection instruction, or for non-preset object faces whose duration has not reached the threshold, delete the cached face data and the corresponding on-camera segments. S11: Determine whether the shooting is complete; if yes, proceed to step S13; otherwise, proceed to step S12. S12: Continue processing the next frame and return to step S4; S13: Determine whether the non-preset object agrees to be in the shot; if yes, proceed to step S14, otherwise proceed to step S18. S14: Displays the face restoration selection interface; S15: In response to a click on the face ID, play the corresponding on-screen clip; S16: Receive user recovery command; S17: Retrieve cached data and replace mosaic data; S18: Generate the final video; S19: Delete all cached data.

[0075] In this embodiment, facial images of preset objects (such as Vlog creators) are acquired before video shooting, features are extracted and encrypted and stored. By establishing an identity whitelist, the preset objects and non-preset objects (such as passersby) can be accurately distinguished, which can solve the problem of indiscriminate blurring in related technologies and ensure that the preset objects are clearly visible in the video without the need for manual adjustment in post-production.

[0076] During video recording, video can be captured in real time and combined with voice and motion information. AI models analyze the data to perform facial comparisons and determine the willingness of passersby to appear in the frame. For each face region in the real-time video identified as not belonging to a pre-defined subject, mosaic processing and caching of the original facial image data are performed. By assigning a unique ID to each passerby and tracking their appearance duration, only passersby who reach a threshold are retained with their cached original data and bound to their unique ID.

[0077] After the video is recorded, the information of passersby is displayed in thumbnails through the recovery selection interface. Users can choose whether to restore the face image of the passerby based on their authorization to appear in the video. If restoration is required, cached data is retrieved based on the ID to replace the mosaic area, achieving reversible restoration to generate the final video.

[0078] After the final video is generated, all cached data is automatically deleted, and a user confirmation step can be added to ensure that the original data is only temporarily stored and destroyed in a timely manner to comply with relevant regulations.

[0079] In summary, the technical solution provided in this application achieves an effective balance between content flexibility and data security, and can better meet the actual creative needs and regulatory requirements for privacy protection.

[0080] like Figure 5 As shown, this application embodiment also provides a video processing apparatus, the video processing apparatus 100 including: Information acquisition module 101 is used to acquire facial feature information of at least one preset object; The face detection module 102 is used to perform face detection on the video frames captured during video shooting, and to determine the face regions in the detected face regions that do not match the face feature information of the preset object as face regions of non-preset objects. Privacy protection module 103 is used to perform privacy protection processing on the face region of each non-preset object in the video; Data caching module 104 is used to cache the original face image data of the non-preset object; The face restoration module 105 is used to restore the face region of the authorized non-preset object in the video segment to the original face image after the video is recorded, in response to the restoration command triggered by the user based on the non-preset object's appearance authorization.

[0081] In some embodiments, the privacy protection module 103 is used for: For each non-preset object in the video, a mosaic effect is applied to the face region of the non-preset object. The size of the mosaic block used in the mosaic effect is negatively correlated with the distance between the non-preset object and the video shooting device.

[0082] In some embodiments, the data caching module 104 is used for: Assign a unique identifier to each detected non-preset object; For each non-preset object, perform the following operations: Record the duration of the non-preset object in the current monitoring window. The current monitoring window starts at the moment when the non-preset object is first detected during the current occurrence process and ends at the moment when the non-preset object is not detected for the first time in a preset number of consecutive frames. The original facial image data and corresponding on-screen clips of the non-preset objects are cached in real time, and the cached data is bound to the unique identifier. When the current monitoring window ends, depending on whether the non-preset object meets the preset conditions, the original face image data of the non-preset object that has been cached and the corresponding on-screen segment are selectively retained for face restoration operation after the video shooting is completed.

[0083] In some embodiments, the non-preset object satisfying the preset conditions includes any one of the following: The on-camera authorization instruction for the non-preset object has been obtained; No authorization confirmation information was obtained regarding whether the non-preset object is allowed to enter the camera, and the non-preset object has been present in the current monitoring window for a preset duration.

[0084] In some embodiments, the apparatus further includes an analysis module, the analysis module being used to: During video recording, based on the voice information and / or behavioral information of the non-preset object, an AI model is used to analyze the non-preset object's authorization to appear on camera.

[0085] In some embodiments, the face restoration module 105 is used for: After the video recording is completed, a recovery selection interface is displayed, in which non-preset objects whose original facial image data has been retained are displayed in thumbnail form; In response to the user's restoration command for an authorized target non-preset object on the restoration selection interface, the system retrieves the original face image data associated with the target non-preset object from the cache, and restores the face region of the target non-preset object in the video segment that has been processed for privacy protection to the original face image.

[0086] In some embodiments, the recovery selection interface has a preview function; the device further includes: The display module is used to respond to the user's preview operation on the recovery selection interface for the target non-preset object, and play at least one on-screen segment of the target non-preset object appearing in the video.

[0087] In some embodiments, the apparatus further includes a data deletion module, the data deletion module being configured to: After generating the final video file based on the face restoration operation on authorized non-preset objects, a data deletion prompt is output for all original face image data in the cache; In response to the user's confirmation of the data deletion prompt, all cached original face image data are deleted; If no confirmation is received from the user within a preset time, all cached original face image data will be automatically deleted.

[0088] It should be noted that the video processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video processing apparatus and video processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0089] This application also provides an electronic device, including a processor, which is configured to invoke instructions to cause the electronic device to execute the steps of the video processing method provided in any of the foregoing embodiments.

[0090] This application also provides a storage medium, including an executable program stored thereon, which, when executed by a processor, implements the steps of the video processing method provided in any of the foregoing embodiments.

[0091] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconstructable. For example, the processor is a hardware circuit implemented by an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconstructable hardware circuit, the processor loads a configuration document and implements a cyclical process of hardware circuit configuration, which can be understood as the processor loading instructions to implement the functions of some or all of the above units or modules in a cyclical process. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.

[0092] The computer-readable storage medium provided in this embodiment can execute the video processing method of the above embodiment. Its implementation principle and technical effect are similar to those of the above embodiment, and will not be repeated here.

[0093] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0094] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0095] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0096] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0097] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A video processing method, characterized in that, The method includes: Obtain facial feature information of at least one preset object; During video recording, face detection is performed on the captured video frames. Face regions that do not match the facial feature information of the preset object are identified as non-preset object face regions. Privacy protection processing is performed on the face regions of each non-preset object in the video, and the original face image data of the non-preset object is cached; After the video recording is completed, in response to the user's restoration command triggered based on the non-preset object's appearance authorization, the cached original face image data is used to restore the face area of ​​the authorized non-preset object in the video, which has been processed for privacy protection, to the original face image.

2. The video processing method according to claim 1, characterized in that, The caching of the original face image data of the non-preset object includes: Assign a unique identifier to each detected non-preset object; For each non-preset object, perform the following operations: Record the duration of the non-preset object in the current monitoring window. The current monitoring window starts at the moment when the non-preset object is first detected during the current occurrence process and ends at the moment when the non-preset object is not detected for the first time in a preset number of consecutive frames. The original facial image data and corresponding on-screen clips of the non-preset objects are cached in real time, and the cached data is bound to the unique identifier. When the current monitoring window ends, depending on whether the non-preset object meets the preset conditions, the original face image data of the non-preset object that has been cached and the corresponding on-screen segment are selectively retained for face restoration operation after the video shooting is completed.

3. The video processing method according to claim 2, characterized in that, The non-preset object meets the preset conditions, including any one of the following: The on-camera authorization instruction for the non-preset object has been obtained; No authorization confirmation information was obtained regarding whether the non-preset object is allowed to enter the camera, and the non-preset object has been present in the current monitoring window for a preset duration.

4. The video processing method according to claim 1, characterized in that, The privacy protection processing performed on the facial regions of each non-preset object in the video includes: For each non-preset object in the video, a mosaic effect is applied to the face region of the non-preset object. The size of the mosaic block used in the mosaic effect is negatively correlated with the distance between the non-preset object and the video shooting device.

5. The video processing method according to claim 1, characterized in that, The method further includes: During video recording, based on the voice information and / or behavioral information of the non-preset object, an AI model is used to analyze the non-preset object's authorization to appear on camera.

6. The video processing method according to claim 1, characterized in that, After video recording is completed, in response to a user's restoration command triggered based on authorization for non-preset subjects to appear in the video, the cached original facial image data is used to restore the privacy-protected facial regions within the authorized non-preset subject segments of the video to their original facial images, including: After the video recording is completed, a recovery selection interface is displayed, in which non-preset objects whose original facial image data has been retained are displayed in thumbnail form; In response to the user's restoration command for an authorized target non-preset object on the restoration selection interface, the system retrieves the original face image data associated with the target non-preset object from the cache, and restores the face region of the target non-preset object in the video segment that has been processed for privacy protection to the original face image.

7. The video processing method according to claim 6, characterized in that, The recovery selection interface has a preview function; the method further includes: In response to the user's preview operation on the recovery selection interface for the target non-preset object, at least one on-screen segment of the target non-preset object appearing in the video is played.

8. The video processing method according to any one of claims 1 to 7, characterized in that, The method further includes: After generating the final video file based on the face restoration operation on authorized non-preset objects, a data deletion prompt is output for all original face image data in the cache; In response to the user's confirmation of the data deletion prompt, all cached original face image data are deleted; If no confirmation is received from the user within a preset time, all cached original face image data will be automatically deleted.

9. A video processing apparatus, characterized in that, The device includes: The information acquisition module is used to acquire facial feature information of at least one preset object; The face detection module is used to perform face detection on the captured video frames during video shooting, and to determine the face regions that do not match the face feature information of the preset object as non-preset object face regions. The privacy protection module is used to perform privacy protection processing on the facial regions of each non-preset object in the video; The data caching module is used to cache the original face image data of the non-preset object; The face restoration module is used to restore the face regions of authorized non-preset objects in the video footage that have been processed for privacy protection to the original face images after the video is recorded, in response to a restoration command triggered by the user based on the authorization of non-preset objects to appear in the video, using the cached original face image data.

10. An electronic device, characterized in that, Includes a processor, the processor being configured to invoke instructions to cause the electronic device to execute the video processing method as described in any one of claims 1 to 8.

Images