Video processing method and device, storage medium and program product

Abstract

The invention discloses a video processing method and device, a storage medium and a program product. Relates to the field of video processing, and the method comprises the steps: obtaining first video data of a target scenic spot, the first video data comprising video data collected by an image collection device of the target scenic spot; a cross-platform audio and video processing tool is utilized to process the first video data to obtain second video data, and the cross-platform audio and video processing tool comprises a tool for format conversion, editing or filter processing of the first video data; source face data is obtained, and the source face data comprises a face image meeting a preset format requirement; based on the source face data, face images in image frames of the second video data are replaced by using a target generative adversarial network to obtain multiple frames of target images; and the target video is determined based on the multiple frames of target images, so that the problem of high difficulty in obtaining tourist video images caused by difficulty in shooting videos in part of scenic spots in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of video processing, and more specifically, to a video processing method, apparatus, storage medium, and program product. Background Technology

[0002] Currently, tourists visiting scenic spots need to record videos and take photos. However, many scenic spots are not suitable for taking pictures (e.g., low visibility, high risk), making it difficult for tourists to obtain the videos they need. In related technologies, it is possible to directly obtain videos of the scenic spot and then use image processing or video editing software to create the images of the scenic spot that tourists need. However, due to the high difficulty in operating video editing or image processing software, it is difficult and inefficient for tourists to obtain the videos they need.

[0003] There is currently no effective solution to the problem that it is difficult to shoot videos in some scenic spots, making it difficult to obtain video images of tourists. Summary of the Invention

[0004] The main objective of this application is to provide a video processing method, apparatus, storage medium, and program product to solve the problem in the related art that it is difficult to shoot and record videos in some scenic spots, resulting in high difficulty in obtaining video images of tourists.

[0005] To achieve the above objectives, according to one aspect of this application, a video processing method is provided. The method includes: acquiring first video data of a target scenic area, wherein the first video data includes video data acquired by an image acquisition device of the target scenic area; processing the first video data using a cross-platform audio and video processing tool to obtain second video data, wherein the cross-platform audio and video processing tool includes tools for format conversion, editing, or filtering of the first video data; acquiring source face data, wherein the source face data includes face images that meet preset format requirements; based on the source face data, replacing the face images in the image frames of the second video data using a target generative adversarial network to obtain multiple target images; and determining a target video based on the multiple target images.

[0006] Further, based on the source face data, a target generative adversarial network is used to replace the face images in the image frames of the second video data to obtain multiple target images, including: extracting image frames from the second video data to obtain multiple first image frames; performing feature extraction on the source face data to obtain source face features; performing feature extraction on each of the multiple first image frames to obtain multiple target face features; performing feature fusion on the source face features and each of the target face features to obtain multiple fused features; and using the target generative adversarial network, adding each of the fused features to the target image frame associated with the fused feature to obtain multiple target images.

[0007] Further, using the target generative adversarial network, each of the fusion features is added to the target image frame associated with the fusion feature to obtain multiple target image frames, including: inputting each of the fusion features into the target generative adversarial network, generating a face image corresponding to the fusion feature using the target generative adversarial network, and fusing the face image corresponding to each of the fusion features into the target image frame associated with the fusion feature to obtain multiple fused target image frames; repairing the fused portion between the face image and the background in each fused target image frame to obtain multiple target image frames.

[0008] Furthermore, the cross-platform audio and video processing tool includes a target encoder, wherein the target encoder uses an image processor for accelerated encoding. The cross-platform audio and video processing tool is used to process the first video data to obtain second video data, including: based on time parameters and a first preset resolution, extracting video segments containing facial images from the first video data to obtain multiple video segments; obtaining encoding parameters and using the target encoder to encode the video images in each video segment to obtain multiple first encoded videos; obtaining an audio encoding format and encoding the audio in each first encoded video to obtain multiple second encoded videos; and splicing and transitioning the multiple second encoded videos to obtain the second video data.

[0009] Further, multiple second-encoded videos are spliced ​​and transitioned to obtain second video data, including: determining the transition type of each second-encoded video; configuring transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; based on the transition type and the transition parameters, using a graphics rendering tool to splice and transition multiple second-encoded videos to obtain spliced ​​first video data, wherein the graphics rendering tool is used to process pixel-level image transitions using the parallel computing power of a graphics processor; verifying the spliced ​​first video data to obtain a first verification result, wherein the verification content for verifying the spliced ​​first video data includes at least one of the following: video playback smoothness, whether there are transition gaps, and whether there is color deviation in the video before and after the transition; after the first verification result indicates that the spliced ​​first video data has passed verification, the spliced ​​first video data is determined as the second video data.

[0010] Furthermore, the cross-platform audio and video processing tool also includes a computer vision library. The cross-platform audio and video processing tool is used to process the first video data to obtain second video data, including: using the cross-platform audio and video processing tool to decode the first video data into image frames to obtain multiple second image frames; using the computer vision library to process each second image frame to obtain multiple processed second image frames, wherein the image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction; encoding the processed multiple second image frames into a video stream to obtain a target video stream; and determining the second video data based on the target video stream.

[0011] Furthermore, after determining the target video based on multiple frames of the target images, the method further includes: verifying the fusion of the face image and the background in the target video to obtain a second verification result; generating an optimization strategy if the second verification result indicates that the target video verification has failed; and replacing the face image in the first video data based on the optimization strategy.

[0012] To achieve the above objectives, according to another aspect of this application, a video processing apparatus is provided. The apparatus includes: a first acquisition unit for acquiring first video data of a target scenic area, wherein the first video data includes video data acquired by an image acquisition device of the target scenic area; a processing unit for processing the first video data using a cross-platform audio-visual processing tool to obtain second video data, wherein the cross-platform audio-visual processing tool includes tools for format conversion, editing, or filtering of the first video data; a second acquisition unit for acquiring source face data, wherein the source face data includes face images that meet preset format requirements; a first replacement unit for replacing face images in image frames of the second video data based on the source face data using a target generative adversarial network to obtain multiple target images; and a determination unit for determining a target video based on the multiple target images.

[0013] Further, the replacement unit includes: a first extraction subunit, used to extract image frames from the second video data to obtain multiple first image frames; a second extraction subunit, used to extract features from the source face data to obtain source face features; a third extraction subunit, used to extract features from each of the multiple first image frames to obtain multiple target face features; a fusion subunit, used to fuse the source face features and each of the target face features to obtain multiple fused features; and an addition subunit, used to use the target generative adversarial network to add each of the fused features to the target image frame associated with the fused feature to obtain multiple target images.

[0014] Further, a subunit is added, including: a generation module, used to input each of the fusion features into a target generative adversarial network, use the target generative adversarial network to generate a face image corresponding to the fusion feature, and fuse the face image corresponding to each of the fusion features into a target image frame associated with the fusion feature to obtain multiple fused target image frames; and a repair unit, used to repair the fused portion between the face image and the background in each fused target image frame to obtain multiple target image frames.

[0015] Furthermore, the cross-platform audio and video processing tool includes: a target encoder, wherein the target encoder uses an image processor for accelerated encoding, and the processing unit includes: a cropping subunit, used to crop video segments containing facial images from the first video data based on time parameters and a first preset resolution, to obtain multiple video segments; a first encoding subunit, used to obtain encoding parameters and use the target encoder to encode the video images in each video segment, to obtain multiple first encoded videos; a first processing subunit, used to obtain an audio encoding format and encode the audio in each first encoded video, to obtain multiple second encoded videos; and a second processing subunit, used to splice and transition the multiple second encoded videos, to obtain second video data.

[0016] Further, the second processing subunit includes: a determining module, used to determine the transition type of each of the second encoded videos; a configuration module, used to configure transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; a processing module, used to perform splicing and transition processing on multiple second encoded videos using a graphics rendering tool based on the transition type and the transition parameters to obtain spliced ​​first video data, wherein the graphics rendering tool is used to process pixel-level image transitions using the parallel computing power of a graphics processor; a verification module, used to verify the spliced ​​first video data to obtain a first verification result, wherein the verification content for verifying the spliced ​​first video data includes at least one of the following: video playback smoothness, whether there are transition gaps, and whether there is color deviation in the video before and after the transition; and a determining module, used to determine the spliced ​​first video data as the second video data after the first verification result indicates that the spliced ​​first video data has passed verification.

[0017] Furthermore, the cross-platform audio and video processing tool also includes a computer vision library. The processing unit includes: a decoding subunit, used to decode the first video data into image frames using the cross-platform audio and video processing tool to obtain multiple second image frames; a third processing subunit, used to process each second image frame using the computer vision library to obtain multiple processed second image frames, wherein the image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction; a second encoding subunit, used to encode the multiple processed second image frames into a video stream to obtain a target video stream; and a determination subunit, used to determine the second video data based on the target video stream.

[0018] Furthermore, the video processing apparatus further includes: a verification unit, configured to verify the fusion of facial images and background in the target video after determining the target video based on multiple frames of the target images, and obtain a second verification result; a generation unit, configured to generate an optimization strategy if the second verification result indicates that the target video verification has failed; and a second replacement unit, configured to replace the facial images in the first video data again based on the optimization strategy.

[0019] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the video processing method.

[0020] According to another aspect of this application, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the video processing method during runtime.

[0021] According to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the video processing method.

[0022] In this application, the following method is adopted: acquiring first video data of the target scenic area, wherein the first video data includes video data acquired by the image acquisition device of the target scenic area; processing the first video data using a cross-platform audio and video processing tool to obtain second video data, wherein the cross-platform audio and video processing tool includes tools for format conversion, editing, or filtering of the first video data; acquiring source face data, wherein the source face data includes face images that meet preset format requirements; based on the source face data, replacing the face images in the image frames of the second video data using a target generative adversarial network to obtain multiple target images; and determining the target video based on the multiple target images, thereby solving the technical problem in related technologies where it is difficult to shoot videos in some scenic areas, resulting in high difficulty in obtaining video images of tourists. By replacing faces in the video of the scenic area frame by frame to obtain video of tourists in the scenic area, the technical effect of reducing the difficulty of obtaining video images of tourists in some scenic areas is avoided, as it is difficult to shoot videos in some scenic areas in related technologies. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1A hardware structure block diagram of a computer terminal for implementing a video processing method is shown.

[0025] Figure 2 This is a flowchart of a video processing method provided according to an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of a video processing apparatus provided according to an embodiment of this application;

[0027] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, video data, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose whether to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose whether to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0031] It should be noted that the face-swapping technology involved in this application must be used under the premise of legality and compliance, and is strictly prohibited from being used for illegal purposes (such as forging other people's identities, spreading false information, infringing on other people's portrait rights, etc.). Before use, authorization of portrait rights of the relevant persons must be obtained, and local laws, regulations and ethical guidelines must be followed.

[0032] Example 1

[0033] According to an embodiment of this application, a method embodiment for video processing is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a video processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the video processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned video processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0038] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0039] Under the aforementioned operating environment, this application provides the following: Figure 2 The video processing method shown. Figure 2 This is a flowchart of a video processing method according to Embodiment 1 of this application.

[0040] Step S201: Obtain the first video data of the target scenic area, wherein the first video data includes: video data collected by the image acquisition device of the target scenic area.

[0041] The aforementioned first video data may include: video data collected by image acquisition equipment in the target scenic area. The image acquisition equipment may be deployed on buildings in the target scenic area, such as iron towers, to collect video of the target scenic area from all directions. The aforementioned first video data may include: video of people's activities.

[0042] In one alternative example, video data can be acquired from the image acquisition device using multiple threads to improve the efficiency of video data acquisition.

[0043] Step S202: Process the first video data using a cross-platform audio and video processing tool to obtain the second video data. The cross-platform audio and video processing tool includes tools for format conversion, editing, or filter processing of the first video data.

[0044] The aforementioned cross-platform audio and video processing tools can be open-source cross-platform audio and video processing tools that support functions such as format conversion, editing, and filter processing. These cross-platform audio and video processing tools can be combined with GPUs (graphics processing units), OpenGL (a cross-platform graphics rendering interface, also known as rendering tools), and OpenCV (a commonly used library in the field of computer vision that supports functions such as image recognition, object detection, and image enhancement) to improve processing efficiency and effects, and meet professional-grade video processing needs.

[0045] In this embodiment, the first video data can be processed using a cross-platform audio and video processing tool to obtain the second video data.

[0046] Step S203: Obtain source face data, wherein the source face data includes: a face image that meets the preset format requirements.

[0047] The aforementioned preset formats may include: supporting a single clear face image (JPG / PNG format, resolution ≥ preset resolution, such as 512×512) or a short video of the target duration, such as a 3-5 second short video (MP4 format).

[0048] The aforementioned source facial data can also meet preset quality requirements, such as unobstructed faces (no masks, glasses, or other obstructions), uniform lighting, and the angle should preferably be frontal or semi-side (avoiding full-side faces that cause pose mismatch). It is recommended to select source data with a face angle similar to that of the target video (corresponding to the second video data) (to reduce the difficulty of subsequent fusion).

[0049] In one alternative example, if video is used as the source data, keyframes can be extracted from it (selecting the frame with the clearest face and the most suitable angle) and saved as a single image (i.e., the source face data) for later use.

[0050] Step S204: Based on the source face data, use a target generative adversarial network to replace the face images in the image frames of the second video data to obtain multiple target images.

[0051] The aforementioned target generative adversarial network can include a mapping network, a generator network, and a discriminator network. The target generative adversarial network can generate face images with natural skin texture, hair details, and lighting transitions, while also supporting fine control of the detailed features of the generated face through style vectors.

[0052] The mapping network, generator network, and discriminator network are explained below:

[0053] Style Mapping:

[0054] 128-dimensional random noise vector (Following a standard normal distribution) Mapped to a 512-dimensional style vector through an 8-layer fully connected network. The formula is:

[0055]

[0056] in This represents the k-th fully connected layer. The purpose of this activation function is to transform noise vectors into semantically meaningful style features, controlling the overall style of the generated face (such as skin color and facial contours).

[0057] Generative Network (Synthesis Network):

[0058] Employing a "progressive resolution upscaling" structure, the image is progressively upgraded from a 4×4 low-resolution feature map to a 1024×1024 high-resolution image. Each resolution upscaling step is achieved through "convolution + upsampling," and the parameters of each convolutional layer are derived from the style vector. control.

[0059] Let the feature map of the i-th resolution be... The formula for the generation process is:

[0060]

[0061] in, yes A subset of (corresponding to the i-th level style). This is an upsampling operation (bilinear interpolation). This is a bias term used to fine-tune the brightness of the feature map. The final target face image is generated through the RGB output layer. (Level 10 corresponds to a resolution of 1024×1024).

[0062] Discriminator Network:

[0063] A progressive resolution reduction structure, symmetrical to the generative network, is used to determine whether the input image is a "real face" or a "generated face." The loss function is WGAN-GP (Wasserstein GAN with Gradient Penalty, a generative adversarial network with gradient-penalized W-distance (bulldozer distance),) with the following formula:

[0064]

[0065] in, For the output of the discriminator, It is a distribution of real facial data. It generates a face distribution. This represents the distribution of differences between real face data and monthly generated face data, where x represents real face data. This represents the generation of face data, where λ is the gradient penalty coefficient (usually set to 10), with the aim of stabilizing the training process and improving the realism of the generated images.

[0066] In this embodiment, the process of replacing the face image can be divided into four stages: "face information extraction - feature fusion - face rendering - video synthesis". Each stage relies on a specific deep learning algorithm. The core formula uses standard mathematical symbols to ensure no garbled characters and clear logic. For example, facial features can be extracted from the source face data, facial features can be extracted from each frame of the second video data, and then the source face data and the facial features of each frame are fused to obtain the fused features of the frame (e.g., feature vector).

[0067] Based on the fused feature vectors, a high-resolution (e.g., 1024×1024) complete face image can be generated, repairing the transition area between the face edge and the target video background, ensuring that the face after face swapping blends naturally with the background without obvious stitching marks (such as "blurred edges" or "inconsistent skin color"). For example, the fused features of each frame can be input into a target generative adversarial network to generate a face image, and then use this face image to replace the original face image in that frame to obtain the target image corresponding to that frame.

[0068] Step S205: Determine the target video based on multiple target images.

[0069] After obtaining multiple target images, these images can be re-encoded into video, and audio from the second video data can be added to ensure audio-video synchronization, thus obtaining the target video.

[0070] For example, the process of re-encoding consecutive frames after face swapping into video and adding the original video's audio to ensure audio-video synchronization is as follows:

[0071] (1) Frame-to-frame video without audio: Based on the frame rate of the original video (i.e. the second video data), the face-swapping frames (multiple target images) in the output directory are sequentially synthesized into a video file without audio. Among them, a suitable encoding format (such as H.264) and image quality parameters (balancing image quality and file size) can be selected.

[0072] (2) Extract the original video audio: Extract the audio track from the preprocessed second video data, keeping the audio format and encoding parameters unchanged (to avoid re-encoding and loss of sound quality).

[0073] (3) Audio and video merging: The synthesized video without audio is merged with the extracted original audio track to ensure that the video duration is consistent with the audio duration, and finally a complete face-swapping video file is generated to obtain the target video.

[0074] Through the above steps, in this embodiment, by replacing faces in a single frame of a video of a scenic area, video of tourists in the scenic area is obtained. This avoids the difficulty in obtaining tourist video images caused by the difficulty of filming in some scenic areas in related technologies, thus achieving the technical effect of reducing the difficulty of obtaining tourist videos in scenic areas. This solves the technical problem in related technologies where filming in some scenic areas is difficult, leading to the high difficulty in obtaining tourist video images.

[0075] Optionally, in the video processing method provided in this application embodiment, based on source face data, a target generative adversarial network is used to replace face images in image frames of second video data to obtain multiple target images, including: extracting image frames from the second video data to obtain multiple first image frames; performing feature extraction on the source face data to obtain source face features; performing feature extraction on each of the multiple first image frames to obtain multiple target face features; performing feature fusion on the source face features and each target face feature to obtain multiple fused features; and using a target generative adversarial network, adding each fused feature to the target image frame associated with the fused feature to obtain multiple target images.

[0076] In one optional example, the second video data may also meet the following preset requirements: (1) Format requirements: MP4 format, resolution is recommended to be ≤ specified resolution, for example, 1080P (too high resolution will significantly increase processing time), duration is ≤ specified duration, for example, 5 minutes (long videos are recommended to be processed in segments to avoid excessive memory usage); (2) Quality requirements: the target face is clear and there is no frequent occlusion (such as frequent head turning or hand occlusion of the face), and the background complexity is moderate (a solid color background is easy to cause unnatural edge blending, and a complex background needs to ensure the accuracy of face detection).

[0077] In this embodiment, the second video data can also be preprocessed: for example, video parameters can be standardized, including resolution (720P or 1080P is recommended), frame rate (25-30fps), and encoding format (to ensure compatibility of subsequent frame extraction and synthesis) to avoid processing abnormalities due to inconsistent parameters.

[0078] In this embodiment, face-swapping processing can be performed on each frame of the second video data, including:

[0079] Video frame extraction: The second video data is decoded into consecutive RGB format image frames and saved to a temporary directory in chronological order to ensure that the frame rate of the frames is consistent with the original video (to avoid fast-forwarding or slow-motion when synthesizing the video later), resulting in multiple first image frames.

[0080] Face detection and alignment: The MTCNN (Multi-task Cascaded Convolutional Neural Network) algorithm is used to locate the target face (i.e., face image) in each frame, output the 5 key points of the face (center of the eyes, tip of the nose, and corners of the mouth on both sides), and rotate and scale the face to a uniform size (224×224) through affine transformation to prepare for feature extraction.

[0081] Feature extraction and fusion: The ArcFace (face image feature extraction) algorithm is used to extract the target dimension features (e.g., 512-dimensional feature vectors) of the source face in the source face data and the target face in the second video data. The AdaIN algorithm (image style transfer algorithm) is used to fuse the two types of features (i.e., source face features and target face features) (preserving the identity features of the source face and inheriting the pose, expression and lighting features of the target face) to generate fused features (e.g., fused feature vectors).

[0082] Facial rendering: The fused feature vector is input into the StyleGAN2 generator network (corresponding to the target generative adversarial network) to generate a face-swapped face that matches the background of the target frame. Then, the transition area between the face edge and the background is repaired by the face segmentation model to ensure that there are no obvious stitching marks.

[0083] Frame saving: Save the processed face-swapped frames to the output directory in their original order, using an uncompressed format (such as PNG) to avoid image quality loss.

[0084] In one alternative example, based on the source face data, a target generative adversarial network is used to replace the face images in the image frames of the second video data to obtain multiple target images. This can rely on a target face-swapping framework, which depends on GPU acceleration and a specific deep learning library. The environment for the target face-swapping framework can be configured by following these steps to avoid version compatibility issues:

[0085] (1) Software environment configuration:

[0086] Install the dedicated graphics card encoding module driver and the parallel computing platform and application programming interface model:

[0087] The driver version must be ≥ the first preset version: Automatically install the recommended driver through the system's built-in driver management tool, and verify the driver status after restarting (the full driver version number and the corresponding parallel computing platform and application programming interface model version must be displayed).

[0088] The version of the parallel computing platform and application programming interface model must be ≥ the second preset version: Download the corresponding version of the parallel computing platform and application programming interface model installation package from the official website, execute the installation process, configure the system environment variables (ensure that the tools related to the parallel computing platform and application programming interface model can be called globally), and finally verify whether the installation of the parallel computing platform and application programming interface model is successful (the correct version information of the parallel computing platform and application programming interface model must be displayed).

[0089] (2) Create a Python virtual environment:

[0090] Download the installation package compatible with your system, follow the instructions to complete the installation, restart the terminal, and then verify whether the Python virtual environment is available.

[0091] Create and activate a dedicated environment: Create a virtual environment with Python 3.9 (this version has the best compatibility with the target face-swapping framework's dependency libraries). After activating the environment, the terminal prefix will display the environment name, and subsequent operations will be performed in this environment.

[0092] (3) Install dependency libraries:

[0093] Deep learning framework: Install a deep learning framework that is compatible with the parallel computing platform and application programming interface model, and ensure that the framework can access GPU resources.

[0094] Computer Vision and Video Processing Libraries: Install the specified versions of OpenCV (for image preprocessing), dlib (for facial landmark detection), and video processing libraries (for frame extraction and synthesis).

[0095] Other basic dependencies: Install numerical computing libraries, scientific computing libraries, image processing libraries, and data transformation tools to ensure compatibility between modules.

[0096] Download the source code and pre-trained model of the target face-swapping framework:

[0097] Obtain the source code: Clone the official source code of the target face-swapping framework through a code hosting platform, and enter the source code directory to prepare for subsequent operations.

[0098] Download the pre-trained model: Download the required model (including ArcFace feature extraction model, face segmentation model, and face swapping model) from the storage link provided by the official target face swapping framework. Place the model files into the "models" folder of the source code according to the specified directory structure, and ensure that the model path is consistent with the program's expectations.

[0099] Optionally, in the video processing method provided in this application embodiment, a target generative adversarial network is used to add each fusion feature to the target image frame associated with the fusion feature to obtain multiple target images. This includes: inputting each fusion feature into the target generative adversarial network, generating a face image corresponding to the fusion feature using the target generative adversarial network, and fusing the face image corresponding to each fusion feature into the target image frame associated with the fusion feature to obtain multiple fused target image frames; repairing the fused portion between the face image in each fused target image frame and the background in the fused target image frame to obtain multiple target images.

[0100] In this embodiment, the fused features can be input into the target generative adversarial network to generate a face-swapped face that matches the background of the target image frame managed by the fused features. Then, the face segmentation model is used to repair the transition area between the face edge (i.e. the face image) and the background to ensure that there are no obvious stitching marks. After that, the processed face-swapped frames (corresponding to multiple target images) can be saved to the output directory in the original order and saved in an uncompressed format (such as PNG) to avoid image quality loss.

[0101] In this embodiment, the deep learning-based video AI face swapping achieves natural and real-time results, and supports replacing a source face with a specified face in the target video.

[0102] Optionally, in the video processing method provided in this application embodiment, the cross-platform audio and video processing tool includes: a target encoder, wherein the target encoder uses an image processor for accelerated encoding, and the cross-platform audio and video processing tool is used to process the first video data to obtain the second video data, including: based on time parameters and a first preset resolution, extracting video segments containing facial images from the first video data to obtain multiple video segments; obtaining encoding parameters, and using the target encoder to encode the video images in each video segment to obtain multiple first encoded videos; obtaining an audio encoding format, and encoding the audio in each first encoded video to obtain multiple second encoded videos; and splicing and transitioning the multiple second encoded videos to obtain the second video data.

[0103] The target encoder utilizes a graphics processing unit (GPU) to accelerate encoding. GPU-accelerated encoding can utilize a dedicated encoding module on the graphics card to replace CPU (Central Processing Unit) encoding, reducing CPU usage while increasing encoding speed (typically by 3-10 times). This dedicated GPU encoding module supports mainstream encoding formats such as H.264, H.265, and AV1, and is compatible with relevant graphics cards. Its core advantage lies in balancing encoding speed, image quality, and file size, making it suitable for batch video processing or real-time encoding scenarios.

[0104] Prerequisites for image processor accelerated encoding: (1) Install relevant drivers that support graphics card dedicated encoding modules: the version must be ≥ the preset version. The driver status can be verified by using the hardware information query command. The complete driver version number and the corresponding parallel computing platform and application programming interface model version must be displayed. (2) Confirm that the cross-platform audio and video processing tool supports graphics card dedicated encoding modules: verify whether the encoder related to graphics card dedicated encoding modules is included by using the encoder query command. The graphics card dedicated encoding modules installed by default may have been pre-compiled to support this. If not, you need to manually compile and enable the corresponding configuration items.

[0105] In this embodiment, the first video data can be processed as follows:

[0106] 1. Video Segment Extraction: Determine the start and end time points of the source video (first video data) (e.g., the start and end time points of the appearance of people, accurate to the second), combine the time parameters (e.g., start and end time points) to extract the target segment, and you can also specify the resolution of the output video (e.g., 1080P, 4K) to ensure that the aspect ratio of the extracted video is consistent with the source video, resulting in multiple video segments.

[0107] 2. Encoding Parameter Configuration: Select the encoding format (H.264 or H.265), configure the encoding speed parameter (divided into multiple levels from fast to slow, the slower the speed, the better the image quality and the smaller the file size) and the image quality control parameter (value range 0-51, recommended 22-28, the smaller the value, the closer the image quality is to the source video), encode the video segment based on the encoding parameters, and obtain multiple first encoded videos.

[0108] 3. Audio processing: In this embodiment, the audio encoding format (such as AAC) can be selected as needed or the source audio can be directly copied (to avoid the loss of sound quality caused by re-encoding), and the audio bit rate can be configured (usually 192kbps can meet the sound quality requirements of high-definition video) to encode the audio in the first encoded video.

[0109] 4. Output and Verification: Specify the output file path, enable the parameter to overwrite existing files (to avoid operation failure due to file name conflicts), output multiple first-encoded videos, and verify the integrity of the picture, picture quality and audio synchronization through a video playback tool after the output is completed.

[0110] Finally, multiple second-encoded videos can be spliced ​​and transitioned to obtain second video data.

[0111] Optionally, in the video processing method provided in this application embodiment, multiple second encoded videos are spliced ​​and transitioned to obtain second video data, including: determining the transition type of each second encoded video; configuring transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; based on the transition type and transition parameters, using a graphics rendering tool to splice and transition the multiple second encoded videos to obtain spliced ​​first video data, wherein the graphics rendering tool is used to process pixel-level image transitions using the parallel computing power of a graphics processor; verifying the spliced ​​first video data to obtain a first verification result, wherein the verification content for verifying the spliced ​​first video data includes at least one of the following: video playback smoothness, whether there are transition gaps, and whether there is color deviation in the video before and after the transition; after the first verification result indicates that the spliced ​​first video data has passed verification, the spliced ​​first video data is determined as second video data.

[0112] The aforementioned image rendering tools can include: cross-platform graphics rendering interfaces; and in cross-platform audio and video processing tools, integrated filter libraries can be used to accelerate image rendering and transition rendering. Compared to CPU transitions, image rendering tools can improve rendering efficiency by more than 50%, while supporting various transition effects such as dissolve, slide, scale, and rotate. The core principle is to utilize the parallel computing power of the GPU to process pixel-level image transitions, reducing image stuttering or blurring and improving the visual smoothness of transitions.

[0113] Prerequisites include: 1. Installing the relevant dependency libraries for image rendering tools: including graphics driver development libraries, GLSL compiler libraries, rendering plugin libraries, etc. These can be installed in batches through system package management tools to ensure that the library file versions are compatible with cross-platform audio and video processing tools.

[0114] 2. Confirm that the cross-platform audio and video processing tool supports image rendering tool filters: Verify whether it includes image rendering tool related filters (such as gltransition (used to describe transition effects implemented by shader language) and glscale (used to implement image transition effects) by using the filter query command. If not supported, you need to manually compile the cross-platform audio and video processing tool and enable the corresponding configuration items.

[0115] The process of stitching and transitioning multiple second-encoded videos to obtain second video data can include the following steps:

[0116] 1. Prepare video segments to be spliced: Ensure that the resolution, frame rate, and encoding format of multiple video segments (corresponding to multiple second-encoded videos) are consistent (if inconsistent, parameters must be unified through cross-platform audio and video processing tools to avoid screen misalignment during transitions). Record the duration of the two video segments (to determine the transition duration). If the resolution and encoding format are inconsistent, convert the resolution and encoding format of multiple second-encoded videos to a unified format.

[0117] 2. Select transition effects and parameters: Choose the transition type according to the video style (such as "dissolve" is suitable for soft scenes, "slide" is suitable for dynamic scenes), and configure the transition duration (usually 1-3 seconds, too short will make the transition abrupt, too long will affect the video rhythm).

[0118] 3. Configure rendering parameters: Specify the image quality level of the graphics rendering tool (such as anti-aliasing parameters to improve the smoothness of transitions), and set the encoding format and bitrate of the output video (to ensure that the overall image quality of the video is consistent after the transition).

[0119] 4. Perform transition rendering and verification: Start the rendering command. After execution, verify the smoothness of the transition effect and whether there are any gaps or color deviations in the picture by playing the frame by frame. If there are any problems, the transition parameters need to be adjusted or the source video parameters need to be unified again to obtain the second video data.

[0120] Optionally, in the video processing method provided in this application embodiment, the cross-platform audio and video processing tool further includes: a computer vision library. The cross-platform audio and video processing tool is used to process the first video data to obtain second video data, including: decoding the first video data into image frames using the cross-platform audio and video processing tool to obtain multiple second image frames; processing each second image frame using the computer vision library to obtain multiple processed second image frames, wherein the image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction; encoding the processed multiple second image frames into a video stream to obtain a target video stream; and determining the second video data based on the target video stream.

[0121] Cross-platform audio and video processing tools can also include computer vision libraries, such as OpenCV (a commonly used library in the field of computer vision), which support functions such as image recognition, object detection, and image enhancement. These tools can call OpenCV's core capabilities through filter interfaces to achieve intelligent video processing (such as face detection, image denoising, and color correction). The core advantage of combining these two approaches is that the cross-platform audio and video processing tools handle video decoding / encoding and stream processing, while OpenCV handles image-level intelligent analysis and processing, forming a complete "stream processing - intelligent analysis - result output" chain.

[0122] Prerequisites: (1) Install OpenCV library: including OpenCV core library and development library, install through system package management tool or source code compilation, and ensure version ≥ 4.0 (supports more computer vision algorithms); (2) Confirm that the cross-platform audio and video processing tool supports OpenCV: verify whether OpenCV related configuration items are enabled by configuration query command (e.g., --enable-libopencv (used to enable video filtering function through OpenCV library)). If not supported, you need to manually compile the cross-platform audio and video processing tool and associate it with the OpenCV library path.

[0123] In one alternative example, the first video data is processed using a cross-platform audio and video processing tool to obtain the second video data, which may include the following processing flow:

[0124] 1. Video Decoding and Frame Extraction: The source video (corresponding to the first video data) is decoded into continuous image frames (such as RGB format) using cross-platform audio and video processing tools, and output sequentially to a temporary directory or memory buffer to ensure that the frame rate of the extracted frames is consistent with that of the source video (to avoid fast-forwarding or slowing down the video).

[0125] 2. OpenCV Intelligent Processing: This function uses OpenCV algorithms to process single or multiple image frames (corresponding to the second image frame), for example:

[0126] (1) Face detection (corresponding to face detection): Locate the face region in the video using Haar (cascade classifier) ​​or deep learning model, and output the face coordinates and size;

[0127] (2) Image denoising: Use algorithms such as Gaussian filtering and median filtering to remove noise from the video (suitable for videos shot in low light).

[0128] (3) Color correction: Adjust the brightness, contrast and saturation of the image to unify the overall color style of the video.

[0129] 3. Post-processing frame encoding: The image frames processed by OpenCV (corresponding to multiple processed second image frames) are re-encoded into a video stream using cross-platform audio and video processing tools. The encoding format (such as H.264) and image quality parameters can also be configured to ensure that the encoded video is synchronized with the source video in terms of duration and audio.

[0130] 4. Result Verification: Play the processed video to verify the intelligent processing effect (such as whether the face detection is accurate, whether the image is clear after noise reduction, and whether the colors are natural). If the effect is not good, you can adjust the OpenCV algorithm parameters (such as the detection threshold and the filter kernel size) to obtain the second video data.

[0131] Optionally, in the video processing method provided in this application embodiment, after determining the target video based on multiple target images, the method further includes: verifying the fusion of the face image and the background in the target video to obtain a second verification result; generating an optimization strategy if the second verification result indicates that the target video verification has failed; and replacing the face image in the first video data based on the optimization strategy.

[0132] In this embodiment, the final generated face-swapped video (corresponding to the target video) can also be played. The key detection items may include, but are not limited to: first, whether the facial features are aligned with the pose of the target frame (no offset or distortion); second, whether the blending of the face edges with the background is natural (no white edges or blur); and third, whether the audio and video are synchronized (no misalignment between the picture and the sound). If there are any problems, the process needs to be returned to the optimization stage to adjust the parameters.

[0133] To avoid the face-swapping effect being affected by data quality, model parameters, and environment configuration, optimization solutions for some issues in this embodiment are shown in Table 1.

[0134] Table 1

[0135]

[0136] Troubleshooting common issues includes:

[0137] 1. Model loading failed (prompt: "Model module not found"):

[0138] Cause: The model file path is incorrect (not placed in the specified directory) or the model file is incomplete (the download was interrupted, causing file corruption).

[0139] Solution: First, confirm that the "models" directory contains the three required subdirectories (corresponding to the ArcFace model, segmentation model, and core face-swapping model, respectively), and that each subdirectory contains the corresponding model file. If the directory structure is complete but an error still occurs, you need to download the model file again from the official link (ensure that the file size is consistent with the official specifications), replace it, and try again.

[0140] 2. Insufficient memory for the parallel computing platform and application programming interface model (prompt: "Parallel computing platform and application programming interface model out of memory"):

[0141] Causes: Insufficient graphics card memory (e.g., using 6GB of graphics memory to process 1080P video), or too many frames processed at once (batch size set too large).

[0142] Solutions: First, reduce the target video resolution (e.g., from 1080P to 720P) to reduce the video memory usage per frame; second, reduce the number of frames processed in batches (e.g., set the batch size to 1-2) to reduce the number of frames processed simultaneously; third, close other processes that consume video memory (e.g., other deep learning programs running in the background), release video memory resources, and then restart the face-swapping process.

[0143] 3. No output after face swapping (output directory is empty):

[0144] Reasons: No face was detected in the target video (face is too small, severely occluded, or at an extreme angle); the detection parameters of the MTCNN model are unreasonable (threshold is too high or minimum face size is set too large).

[0145] Solution: First, check the original video frames in the temporary directory to confirm whether there is a face in the frame (if there is no face, the target video needs to be replaced); if there is a face in the frame but it is not detected, the parameters of the MTCNN algorithm need to be adjusted (such as lowering the detection threshold and reducing the minimum face size) to improve the detection sensitivity, and the face swapping process needs to be executed again.

[0146] 4. Audio desynchronization (video fast-forwarding or slow-motion):

[0147] Cause: The frame rate when compositing the video is inconsistent with the frame rate of the original video (e.g., the original video is 25fps, but 30fps is used during compositing), resulting in a change in the video duration.

[0148] Solution: First, check the actual frame rate of the original video (obtained through a video information tool), then adjust the frame rate parameters when merging the video to ensure consistency with the original video; if synchronization problems occur in the merged video, it is necessary to re-merge it at the correct frame rate, or adjust the audio and video synchronization relationship through a video editing tool.

[0149] It should be noted that in this embodiment, long videos can be processed in segments (3-5 minutes each) to avoid excessive memory and video memory usage. If multiple videos need to be processed in batches, they can be executed sequentially using a task scheduling tool to avoid resource contention caused by starting multiple processes simultaneously. Temporary directories (such as frame files and log files) should be cleaned up regularly to free up storage space.

[0150] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0151] Example 2

[0152] This application also provides a video processing apparatus. It should be noted that the video processing apparatus of this application can be used to execute the video processing method provided in this application. The video processing apparatus provided in this application will be described below.

[0153] According to embodiments of this application, an apparatus for implementing the above-described video processing method is also provided, such as... Figure 3 As shown, the device includes: a first acquisition unit 31, a processing unit 32, a second acquisition unit 33, a first replacement unit 34, and a determination unit 35.

[0154] The first acquisition unit 31 is used to acquire the first video data of the target scenic area, wherein the first video data includes: video data acquired by the image acquisition device of the target scenic area;

[0155] Processing unit 32 is used to process the first video data using cross-platform audio and video processing tools to obtain second video data. The cross-platform audio and video processing tools include tools for format conversion, editing, or filter processing of the first video data.

[0156] The second acquisition unit 33 is used to acquire source face data, wherein the source face data includes: a face image that meets the preset format requirements;

[0157] The first replacement unit 34 is used to replace the face images in the image frames of the second video data based on the source face data using a target generative adversarial network to obtain multiple frames of target images.

[0158] The determination unit 35 is used to determine the target video based on multiple frames of target images.

[0159] In the video processing apparatus provided in this application embodiment, the first acquisition unit 31 can acquire first video data of the target scenic area, wherein the first video data includes video data acquired by the image acquisition device of the target scenic area. The processing unit 32 is used to process the first video data using a cross-platform audio and video processing tool to obtain second video data. The cross-platform audio and video processing tool includes a tool for format conversion, editing or filtering of the first video data. The second acquisition unit 33 acquires source face data, wherein the source face data includes face images that meet preset format requirements. The first replacement unit 34 replaces the face images in the image frames of the second video data based on the source face data using a target generative adversarial network to obtain multiple target images. The determination unit 35 is used to determine the target video based on the multiple target images, thereby solving the technical problem in the related technology that it is difficult to shoot and record in some scenic areas, resulting in high difficulty in obtaining video images of tourists. In this embodiment, by replacing faces in a video of a scenic area frame by frame, a video of tourists in the scenic area can be obtained. This avoids the difficulty of obtaining tourist video images caused by the difficulty of filming and recording in some scenic areas in related technologies, thereby achieving the technical effect of reducing the difficulty of obtaining tourist videos in scenic areas.

[0160] Optionally, in the video processing apparatus provided in this application embodiment, the replacement unit includes: a first extraction subunit, used to extract image frames from the second video data to obtain a plurality of first image frames; a second extraction subunit, used to extract features from the source face data to obtain source face features; a third extraction subunit, used to extract features from each of the plurality of first image frames to obtain a plurality of target face features; a fusion subunit, used to fuse the source face features and each target face feature to obtain a plurality of fused features; and an addition subunit, used to use a target generative adversarial network to add each fused feature to the target image frame associated with the fused feature to obtain a plurality of target images.

[0161] Optionally, in the video processing apparatus provided in this application embodiment, a subunit is added, including: a generation module, used to input each fusion feature into a target generative adversarial network, generate a face image corresponding to the fusion feature using the target generative adversarial network, and fuse the face image corresponding to each fusion feature into a target image frame associated with the fusion feature to obtain multiple fused target image frames; and a repair unit, used to repair the fused portion between the face image in each fused target image frame and the background in the fused target image frame to obtain multiple target images.

[0162] Optionally, in the video processing apparatus provided in this application embodiment, the cross-platform audio and video processing tool includes: a target encoder, wherein the target encoder uses an image processor for accelerated encoding, and the processing unit includes: a cropping subunit, used to crop video segments containing facial images from first video data based on time parameters and a first preset resolution, to obtain multiple video segments; a first encoding subunit, used to obtain encoding parameters and use the target encoder to encode the video images in each video segment, to obtain multiple first encoded videos; a first processing subunit, used to obtain an audio encoding format and encode the audio in each first encoded video, to obtain multiple second encoded videos; and a second processing subunit, used to splice and transition the multiple second encoded videos to obtain second video data.

[0163] Optionally, in the video processing apparatus provided in this application embodiment, the second processing subunit includes: a determining module, used to determine the transition type of each second coded video; a configuration module, used to configure transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; a processing module, used to perform splicing and transition processing on multiple second coded videos using a graphics rendering tool based on the transition type and transition parameters to obtain spliced ​​first video data, wherein the graphics rendering tool is used to process pixel-level image transitions using the parallel computing power of a graphics processor; a verification module, used to verify the spliced ​​first video data to obtain a first verification result, wherein the verification content for verifying the spliced ​​first video data includes at least one of the following: video playback smoothness, whether there are transition gaps, and whether there is color deviation in the video before and after the transition; and a determining module, used to determine the spliced ​​first video data as second video data after the first verification result indicates that the spliced ​​first video data has passed verification.

[0164] Optionally, in the video processing apparatus provided in this application embodiment, the cross-platform audio and video processing tool further includes: a computer vision library; the processing unit includes: a decoding subunit, used to decode the first video data into image frames using the cross-platform audio and video processing tool to obtain a plurality of second image frames; a third processing subunit, used to process each second image frame using the computer vision library to obtain a plurality of processed second image frames, wherein the image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction; a second encoding subunit, used to encode the plurality of processed second image frames into a video stream to obtain a target video stream; and a determination subunit, used to determine the second video data based on the target video stream.

[0165] Optionally, in the video processing apparatus provided in the embodiments of this application, the video processing apparatus further includes: a verification unit, configured to verify the fusion of face images and background in the target video after determining the target video based on multiple frames of target images, and obtain a second verification result; a generation unit, configured to generate an optimization strategy if the second verification result indicates that the target video verification has failed; and a second replacement unit, configured to replace the face images in the first video data again based on the optimization strategy.

[0166] It should be noted that the first acquisition unit 31, processing unit 32, second acquisition unit 33, first replacement unit 34, and determining unit 35 mentioned above correspond to steps S201 to S205 in Embodiment 1. Each unit and its corresponding step implement the same instance and application scenario, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0167] Example 3

[0168] Embodiments of this application may provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0169] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0170] The processor can access information and applications stored in the memory via a transmission device to perform the following steps: acquiring first video data of the target scenic area, wherein the first video data includes video data acquired by the image acquisition device of the target scenic area; processing the first video data using a cross-platform audio and video processing tool to obtain second video data, wherein the cross-platform audio and video processing tool includes tools for format conversion, editing, or filtering of the first video data; acquiring source face data, wherein the source face data includes face images that meet preset format requirements; based on the source face data, replacing the face images in the image frames of the second video data using a target generative adversarial network to obtain multiple target images; and determining the target video based on the multiple target images.

[0171] The processor can also invoke information and applications stored in the memory via a transmission device to perform the following steps: based on source face data, replace face images in image frames of second video data using a target generative adversarial network to obtain multiple target images, including: extracting image frames from the second video data to obtain multiple first image frames; extracting features from the source face data to obtain source face features; extracting features from each of the multiple first image frames to obtain multiple target face features; fusing the source face features and each target face feature to obtain multiple fused features; and using a target generative adversarial network to add each fused feature to the target image frame associated with that fused feature to obtain multiple target images.

[0172] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: using a target generative adversarial network (PGAN), adding each fusion feature to the target image frame associated with the fusion feature to obtain multiple target images, including: inputting each fusion feature into the PGAN, generating a face image corresponding to the fusion feature using the PGAN, and fusing the face image corresponding to each fusion feature into the target image frame associated with the fusion feature to obtain multiple fused target image frames; repairing the fused portion between the face image and the background in each fused target image frame to obtain multiple target images.

[0173] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: The cross-platform audio and video processing tool includes a target encoder, wherein the target encoder uses an image processor for accelerated encoding. The cross-platform audio and video processing tool processes the first video data to obtain the second video data, including: based on time parameters and a first preset resolution, extracting video segments containing facial images from the first video data to obtain multiple video segments; obtaining encoding parameters and encoding the video images in each video segment using the target encoder to obtain multiple first encoded videos; obtaining an audio encoding format and encoding the audio in each first encoded video to obtain multiple second encoded videos; and splicing and transitioning the multiple second encoded videos to obtain the second video data.

[0174] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: splicing and transitioning multiple second coded videos to obtain second video data, including: determining the transition type of each second coded video; configuring transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; based on the transition type and transition parameters, using a graphics rendering tool to splice and transition multiple second coded videos to obtain spliced ​​first video data, wherein the graphics rendering tool is used to process pixel-level image transitions using the parallel computing power of the graphics processor; verifying the spliced ​​first video data to obtain a first verification result, wherein the verification content for verifying the spliced ​​first video data includes at least one of the following: video playback smoothness, whether there are transition gaps, and whether there is color deviation in the video before and after the transition; after the first verification result indicates that the spliced ​​first video data has passed verification, the spliced ​​first video data is determined as the second video data.

[0175] The processor can also invoke information and applications stored in the memory via a transmission device to perform the following steps: The cross-platform audio and video processing tool further includes a computer vision library. The cross-platform audio and video processing tool is used to process the first video data to obtain second video data, including: decoding the first video data into image frames using the cross-platform audio and video processing tool to obtain multiple second image frames; processing each second image frame using the computer vision library to obtain multiple processed second image frames, wherein the image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction; encoding the processed multiple second image frames into a video stream to obtain a target video stream; and determining the second video data based on the target video stream.

[0176] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: after determining the target video based on multiple target images, the processor further includes: verifying the fusion of the face image and the background in the target video to obtain a second verification result; if the second verification result indicates that the target video verification has failed, generating an optimization strategy; and replacing the face image in the first video data based on the optimization strategy.

[0177] By employing the embodiments of this application, single-frame replacement of faces in the video of a scenic area is performed to obtain videos of tourists in the scenic area. This avoids the difficulty in obtaining video images of tourists in some scenic areas, which is difficult to record in related technologies, thereby achieving the technical effect of reducing the difficulty of obtaining videos of tourists in scenic areas.

[0178] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0179] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0180] Example 4

[0181] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the video processing method provided in Embodiment 1.

[0182] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0183] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform video processing method steps.

[0184] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0185] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0190] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A video processing method, characterized in that, include: Acquire first video data of the target scenic area, wherein the first video data includes: video data acquired by the image acquisition device of the target scenic area; The first video data is processed using a cross-platform audio and video processing tool to obtain the second video data. The cross-platform audio and video processing tool includes tools for format conversion, editing, or filter processing of the first video data. Acquire source facial data, wherein the source facial data includes: facial images that meet preset format requirements; Based on the source face data, a target generative adversarial network is used to replace the face images in the image frames of the second video data to obtain multiple target images; The target video is determined based on multiple frames of the target image.

2. The video processing method according to claim 1, characterized in that, Based on the source face data, a target generative adversarial network is used to replace the face images in the image frames of the second video data to obtain multiple target images, including: Extract image frames from the second video data to obtain multiple first image frames; Feature extraction is performed on the source face data to obtain source face features; Feature extraction is performed on each of the multiple first image frames to obtain multiple target facial features; The source facial features and each of the target facial features are fused to obtain multiple fused features; Using the target generative adversarial network, each of the fusion features is added to the target image frame associated with the fusion feature to obtain multiple frames of the target image.

3. The video processing method according to claim 2, characterized in that, Using the target generative adversarial network, each of the fusion features is added to the target image frame associated with that fusion feature to obtain multiple frames of the target image, including: Each of the fusion features is input into the target generative adversarial network, the target generative adversarial network is used to generate the face image corresponding to the fusion feature, and the face image corresponding to each of the fusion features is fused into the target image frame associated with the fusion feature to obtain multiple fused target image frames; The portion of the face image and the background in each fused target image frame is repaired to obtain multiple target images.

4. The video processing method according to claim 1, characterized in that, The cross-platform audio and video processing tool includes a target encoder, wherein the target encoder employs image processor-accelerated encoding, and the cross-platform audio and video processing tool processes the first video data to obtain second video data, including: Based on time parameters and a first preset resolution, video segments containing facial images are extracted from the first video data to obtain multiple video segments; Obtain encoding parameters, and use the target encoder to encode the video images in each video segment to obtain multiple first encoded videos; Obtain the audio encoding format, encode the audio in each of the first encoded videos, and obtain multiple second encoded videos; Multiple second-encoded videos are spliced ​​and transitioned to obtain the second video data.

5. The video processing method according to claim 4, characterized in that, Multiple second-encoded videos are spliced ​​and transitioned to obtain second video data, including: Determine the transition type for each of the second encoded videos; Configure transition parameters, wherein the transition parameters include at least one of the following: image quality level, encoding format, and transition duration; Based on the transition type and the transition parameters, multiple second-encoded videos are spliced ​​and transitioned using a graphics rendering tool to obtain spliced ​​first video data. The graphics rendering tool is used to process pixel-level screen transitions using the parallel computing power of the graphics processor. The first video data after splicing is verified to obtain a first verification result. The verification content for verifying the first video data after splicing includes at least one of the following: the smoothness of video playback, whether there are gaps in the transition, and whether there is color deviation in the video before and after the transition. After the first verification result indicates that the spliced ​​first video data has passed verification, the spliced ​​first video data is determined as the second video data.

6. The video processing method according to claim 4, characterized in that, The cross-platform audio and video processing tool also includes: a computer vision library, which processes the first video data using the cross-platform audio and video processing tool to obtain second video data, including: Using the cross-platform audio and video processing tool, the first video data is decoded into image frames to obtain multiple second image frames; Each second image frame is processed using the computer vision library to obtain multiple processed second image frames. The image processing strategy using the computer vision library includes at least one of the following: face detection, image denoising, and color difference correction. The processed multiple second image frames are encoded into a video stream to obtain the target video stream; The second video data is determined based on the target video stream.

7. The video processing method according to claim 1, characterized in that, After determining the target video based on multiple frames of the target images, the process further includes: The fusion of the facial image and the background in the target video is verified to obtain a second verification result; If the second verification result indicates that the target video verification has failed, an optimization strategy is generated; Based on the optimization strategy, the facial images in the first video data are replaced again.

8. A video processing apparatus, characterized in that, include: The first acquisition unit is used to acquire first video data of the target scenic area, wherein the first video data includes: video data acquired by the image acquisition device of the target scenic area; The processing unit is used to process the first video data using a cross-platform audio and video processing tool to obtain the second video data, wherein the cross-platform audio and video processing tool includes: a tool for format conversion, editing or filtering the first video data; The second acquisition unit is used to acquire source face data, wherein the source face data includes: a face image that meets the preset format requirements; The first replacement unit is used to replace the face images in the image frames of the second video data based on the source face data using a target generative adversarial network to obtain multiple frames of target images. The determining unit is used to determine the target video based on multiple frames of the target image.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the video processing method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the video processing method according to any one of claims 1 to 7.

Images