Video processing method and device, electronic equipment and storage medium

By reviewing the image quality of the input video and generating output video frames that meet the standards, the problem of poor video material quality is solved, and high-quality and continuous output video generation is achieved.

CN119071536APending Publication Date: 2024-12-03SHANGHAI BILIBILI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411081296.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing video processing technology is difficult to effectively review and improve the image quality of video materials, resulting in poor quality of new videos generated, especially when the input video is not enough, the continuity and quality of the output video cannot be guaranteed.

Method used

By obtaining the image frames in the input video, determine the qualified frames that meet the image quality standards, and generate the output video. If the output video is insufficient, a supplementary frame is generated based on the output image frame and convert it into the output video to ensure the duration and quality of the output video. .

Benefits of technology

The image quality of the generated new video is improved, ensuring that the output video is sufficient and the quality is qualified, avoiding waste of computing resources, and improving the efficiency of the video generation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071536A_ABST
    Figure CN119071536A_ABST
Patent Text Reader

Abstract

The invention provides a video processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. According to the implementation scheme, an input video is obtained, wherein the input video comprises at least one input image frame and an input audio; determining, from the at least one input image frame, at least one qualified input frame having image content conforming to an image quality criterion; generating an output video, the output video comprising at least one output image frame corresponding to the qualified input frame, where an output audio of the output video is the same as the input audio, and the step of generating the output video comprises: generating at least one supplementary image frame based on the at least one output image frame under the condition that the duration of the image sequence of the at least one output image frame is less than the duration of the output audio, and converting the at least one output image frame and the at least one supplemental image frame into the output video.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a video processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] On current video platforms, users can upload video material and generate new videos containing virtual images based on templates provided by the platform.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or used. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a data processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a video processing method is provided, comprising: acquiring an input video, the input video comprising at least one input image frame and input audio; determining at least one qualified input frame having image content that meets an image quality standard from the at least one input image frame; generating an output video, the output video comprising at least one output image frame corresponding to the qualified input frame, wherein the output audio of the output video is the same as the input audio, and the generating of the output video comprises: when the duration of an image sequence of the at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on the at least one output image frame, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

[0006] According to another aspect of the present disclosure, a video processing device is also provided, including: an input unit, configured to obtain an input video, the input video including at least one input image frame and input audio; an analysis unit, configured to determine at least one qualified input frame having image content that meets image quality standards from the at least one input image frame; a generation unit, generating an output video, the output video including at least one output image frame corresponding to the qualified input frame, wherein the output audio of the output video is the same as the input audio, and the generating of the output video includes: when the duration of the image sequence of the at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on the at least one output image frame, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

[0007] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, and the computer program implements the above method when executed by the at least one processor.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing a computer program is also provided, wherein the computer program implements the above method when executed by a processor.

[0009] According to another aspect of the present disclosure, a computer program product is further provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0010] By using the embodiments provided by the present disclosure, it is possible to check whether the image quality of the input video is qualified, thereby improving the image quality of the new output video generated based on the input video. In the case where the duration of the output video is insufficient, an output video of sufficient duration is generated by multiplexing the already generated output video frames of qualified quality.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 An exemplary flow chart of a video processing method according to an embodiment of the present disclosure is shown;

[0014] Figure 2 An exemplary block diagram of a video processing device according to an embodiment of the present disclosure is shown;

[0015] Figure 3 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0016] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0017] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0018] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0019] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0020] In order to improve the quality of a new video generated according to video materials uploaded by a user, the present disclosure provides a new video processing method.

[0021] Figure 1 An exemplary flow chart of a video processing method according to an embodiment of the present disclosure is shown.

[0022] In step S102, an input video is obtained, where the input video includes at least one input image frame and input audio.

[0023] In step S104, at least one qualified input frame having image content meeting the image quality standard is determined from at least one input image frame.

[0024] In step S106, an output video is generated, the output video includes at least one output image frame corresponding to a qualified input frame, wherein the output audio of the output video is the same as the input audio, and generating the output video includes: when the duration of the image sequence of at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on at least one output image frame, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

[0025] By using the video processing method provided by the embodiment of the present disclosure, it is possible to check whether the image quality of the input video is qualified, thereby improving the image quality of the new output video generated based on the input video. In the case where the output video duration is insufficient, an output video of sufficient duration is generated by multiplexing the already generated output video frames of qualified quality.

[0026] In some applications, users can upload the material they shot to a server, and then generate a new video corresponding to the material based on the selected template. An example of the above application is a digital human video. Users can record video material containing a character image of a content broadcast, and use the above video material and a digital human template to generate a new video broadcast by a digital human. The model can be used to drive the digital human to imitate the movements and lip shapes of the characters in the video material to make the digital human video more natural. Without departing from the principles of the present disclosure, the applications involved in the embodiments of the present disclosure are not limited to digital human videos. The video processing method provided by the present disclosure can also be used for other types of video generation.

[0027] The principles of the present disclosure will be described in detail below.

[0028] In step S102, an input video may be obtained.

[0029] The input video may include an image sequence formed by at least one input image frame, and input audio corresponding to the image sequence of the input video. In an example, the content of the input video may be a person giving a speech, the input image frame includes an image sequence showing the speech process, and the input audio includes the voice content of the speech. The audio duration of the voice content and the duration of the image sequence may be equal. In some examples, the input video may be a video obtained by shooting a real person in the real world. In other examples, the input video may also be a video obtained by any shooting or generation means.

[0030] In step S104, at least one qualified input frame having image content meeting the image quality standard is determined from at least one input image frame.

[0031] Before generating a new video using an input video, an image analysis may be performed on image frames in the input video to determine whether the input video meets the image requirements for generating the new video. In some examples, image analysis may be performed frame by frame on at least one input image frame in the input video. In other examples, at least one input image frame may be sampled, and image analysis may be performed only on the sampled input image frames.

[0032] In some embodiments, the image quality standard used in the embodiments of the present disclosure may include at least one of the following: whether the light in the image falls within a preset brightness range, whether the angle of head shaking of the face in the image falls within a preset angle range, whether the number of faces in the image falls within a preset number range, and whether the proportion of the size of the face in the image falls within a preset proportion range.

[0033] The preset brightness range may indicate whether the light in the video is overexposed, underexposed, or poorly lit. The above light problems may cause the image quality in the image to be lower, and thus cause the image quality of the output video generated based on the input image to be lower.

[0034] The preset angle range of the shaking of the face and head in the image can be used to evaluate whether the character's movements in the video are too large to affect the recognition of the character's movements in the picture. For example, when the angle of the face shaking exceeds the preset angle range (such as greater than 30 degrees), it may be difficult to recognize the character's lip shape in the picture, thereby affecting the lip shape simulation of the virtual character in the output video.

[0035] The number of faces in the image indicates the number of human images that appear in the picture. In some examples, the preset number range can be set to 1. That is, there should be only one human image in the picture of the input video. This will make the actions of the virtual image in the generated output video come from the same person, without being disturbed by the actions of other people.

[0036] The ratio of the size of the face in the image to the screen can reflect the image quality of the character in the video. In some embodiments, the predetermined ratio range can be greater than 1 / 8, that is, the proportion of the face in the screen is greater than 1 / 8. A larger face ratio can provide more detailed lip shape information, thereby improving the quality of the virtual image in the output video.

[0037] The related art at most considers whether there is a required character image (for example, whether there is a face) in the image, but does not consider the impact of the size and movement amplitude of the face in the image on the image quality.

[0038] In some embodiments, image analysis can be performed on the image frames of the input video based on the above-mentioned image quality standards. The image frames of the input video that meet the image quality standards will be determined to be qualified input frames. If the number of frames in the image frames in the input video that do not meet at least one of the above-mentioned image quality standards is greater than a predetermined qualified threshold (such as 10%), the input video can be determined to be unqualified material. Therefore, when the number of qualified input frames is less than the predetermined qualified threshold, the generation of the output video is abandoned. Using the above method, the picture quality of the input video can be automatically analyzed effectively, and the video generation process can be prevented from being still run when the quality of the input video is poor, resulting in a waste of computing resources.

[0039] In step S106, an output video is generated, where the output video includes at least one output image frame corresponding to a qualified input frame, wherein the output audio of the output video is the same as the input audio.

[0040] The input video can be processed using various trained models (such as machine learning models, image feature extraction models, etc.) to extract image features that need to be reproduced in the output video, and generate corresponding output image frames based on the extracted image features. In some embodiments, the entire input video can be used as a model input for extracting image features. In other embodiments, only qualified input frames in the input video can be used as model input for extracting image features.

[0041] At least one output image frame can be generated based on the actions of the characters in the input video (in the qualified input frame), wherein the virtual image in the at least one output image frame also performs the corresponding actions. In some examples, the actions performed by the characters may include body movements, and may also include lip movements, facial expressions, etc. Using the above method, some image information in the input video can be reproduced in the output video. For example, the virtual character in the output video can reproduce the lip shape and actions of the character in the input video to match the content of the input audio.

[0042] The process of generating an output video in step S106 may include: when the duration of the image sequence of at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on at least one output image frame, and converting at least one output image frame and at least one supplementary image frame into an output video.

[0043] In some embodiments, the quality of at least one generated output image frame can be evaluated. For example, at least one generated output image frame can be extracted, and it is determined whether the extracted output image frame meets the aforementioned image quality standard to exclude unqualified output image frames. The above method can be used to implement frame extraction in a simple manner, which is suitable for scenes where the video content does not change drastically. In some examples, the frame extraction can be performed by extracting one frame per second. In other examples, the frame extraction can be performed in rounds using a binary method. In the first round, the output image frame N0 at the bisection position on the output video time axis can be extracted, and the second round can be based on the output image frame extracted in the first round as a boundary, and the output image frame N1 at the bisection position between the output video start point and N0 and the output image frame N2 at the bisection position between N0 and the video end point can be extracted. By analogy, the output video can be continuously divided into two equal parts and the corresponding image frames can be extracted for quality evaluation. The above method can effectively capture representative frames of different time periods and avoid redundant frame extraction. In the example, the frames extracted from a single video can be packaged into a zip package and uploaded to the server. The server side needs to parse all the images in the zip package and input them into the machine processing model used to review the image quality to obtain the review results.

[0044] After quality evaluation, some output image frames with unqualified image quality can be excluded from the output video, resulting in the duration of the output image frames being shorter than the duration of the audio to be output. This is because the source of information when generating the output video is the input video, so at most an output video with the same duration as the input video is generated. After excluding the output image frames with unqualified quality, the duration of the output video may be shorter than the output audio (the output audio is the same as the input audio). In some examples, the unqualified output image frames can be directly removed. In other embodiments, a section of the output image sequence including at least one unqualified output video frame can be removed.

[0045] In order to solve the above problems and ultimately provide a qualified output video, the present disclosure provides a method for generating a supplementary image frame using a qualified output image frame, thereby being able to fill the duration of the output video.

[0046] In some embodiments, at least one supplementary image frame is identical to a portion of at least one output image frame. In some examples, at least a portion of a qualified output image frame may be duplicated, and the duration of the output video may be increased in this manner. In an example, an image sequence formed by at least one output image frame may be reversed, and at least one output image frame and the reversed image sequence may be merged to obtain an output video. The reversed image sequence may be arranged after the qualified output image frame is output. In this manner, the duration of the output video may be filled, and the viewer may not be aware of the presence of repeated content in the video, thereby improving the viewing effect of the video.

[0047] In the example, a user uploads a material video. The server performs frame extraction analysis or full analysis on the uploaded video. The quality review of the image frame is performed using a model executed by a pre-configured processor, and the content of the quality review includes at least one of the following items:

[0048] Light changes: determine whether the light in the video is overexposed, underexposed, or poorly lit;

[0049] Head angle: detects whether the head shaking is greater than 30 degrees;

[0050] Number of faces: The number of faces in the video should be 1;

[0051] Face ratio: Determines whether faces account for 1 / 8 of the video.

[0052] If the above four audit results account for >= 10% of the total number of frames, the audit will fail. If the above four audit results account for < 10% of the total number of frames, the audit will pass.

[0053] The output video can be generated using the approved input video. The output video can be generated using the image information in the input video according to the application scenario. The generated output video can be frame extracted and image analysis can be performed on the extracted image frames to exclude image frames with unqualified image quality in the output video.

[0054] If there are no unqualified image frames in the output video, the starting point of the input video can be used as the starting point of the output video. In the example, the beginning and end of the output video can be deleted according to the actual situation to meet the application needs of the century.

[0055] If there are image frames of unqualified quality in the output video, the unqualified image frames can be excluded and the available image frames of qualified quality can be retained. If there are multiple sequences of available image frames, the sequences with longer available durations are retained to a limited extent. After the playback of the sequence of available image frames is completed, if the duration of the image sequence has not reached the audio duration, the sequence of available image frames can be played in reverse order so that the output video has sufficient output duration. In the case of multiple sequences of available image frames, when the first sequence of available image frames is played, the first sequence of available image frames can be played in reverse order until the starting point of the next sequence of available image frames. Similarly, the insufficient duration caused by excluding image frames of unqualified quality can be filled by playing in reverse order.

[0056] Figure 2 An exemplary block diagram of a video processing device according to an embodiment of the present disclosure is shown.

[0057] like Figure 2As shown, the video processing apparatus 200 may include an input unit 210 , an analysis unit 220 , and a generation unit 230 .

[0058] The input unit 210 may be configured to acquire an input video, the input video comprising at least one input image frame and input audio. The analysis unit 220 may be configured to determine at least one qualified input frame having image content that meets the image quality standard from the at least one input image frame. The generation unit 230 may be configured to generate an output video, the output video comprising at least one output image frame corresponding to the qualified input frame, wherein the output audio of the output video is the same as the input audio, and generating the output video comprises: generating at least one supplementary image frame based on the at least one output image frame when the duration of the image sequence of the at least one output image frame is less than the duration of the output audio, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

[0059] In some embodiments, at least one supplemental image frame is identical to a portion of at least one output image frame.

[0060] In some embodiments, converting at least one output image frame and at least one supplementary image frame into an output video includes: reversing the arrangement order of an image sequence formed by at least one output image frame to obtain a reversed image sequence, and merging at least one output image frame and the reversed image sequence to obtain an output video.

[0061] In some embodiments, generating the output video further includes: extracting an output image frame from the at least one generated output image frame; and determining whether the extracted output image frame meets an image quality standard to exclude unqualified output image frames.

[0062] In some embodiments, the image quality standard includes at least one of the following: whether the light in the image falls within a preset brightness range; whether the head shaking angle of the face in the image falls within a preset angle range; whether the number of faces in the image falls within a preset number range; whether the proportion of the size of the face in the image falls within a preset proportion range.

[0063] In some embodiments, when the number of qualified input frames is less than a predetermined qualified threshold, generating the output video is abandoned.

[0064] In some embodiments, at least one output image frame includes a predetermined virtual image, and generating an output video including at least one output image frame corresponding to a qualified input frame includes: generating at least one output image frame based on the actions of a character in the qualified input frame, wherein the virtual image in at least one output image frame performs an action.

[0065] In some embodiments, the motions include body movements and lip movements.

[0066] It should be understood that Figure 2 The various units of the device 200 shown in FIG. 2 can be used in conjunction with the reference Figure 1 The steps in the method 100 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 100 are also applicable to the device 200 and the units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0067] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 3 The various units described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of the input unit 210, the analysis unit 220, and the generation unit 230 can be implemented together in a system on chip (System on Chip, SoC). SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.

[0068] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, and the computer program implements the above method when executed by the at least one processor.

[0069] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing a computer program is also provided, wherein the computer program implements the above method when executed by a processor.

[0070] According to another aspect of the present disclosure, a computer program product is further provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0071] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0072] See also Figure 3 , a block diagram of an electronic device 300 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device can be a computer device of different types, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0073] like Figure 3 As shown, the electronic device 300 may include at least one processor 301 , a working memory 302 , an input unit 304 , a display unit 305 , a speaker 306 , a storage unit 307 , a communication unit 308 , and other output units 309 that can communicate with each other via a system bus 303 .

[0074] The processor 301 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. The processor 301 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. The processor 301 may be configured to obtain and execute computer-readable instructions stored in the working memory 302, the storage unit 307, or other computer-readable media, such as program codes of an operating system 302a, program codes of an application 302b, and the like.

[0075] The working memory 302 and the storage unit 307 are examples of computer-readable storage media for storing instructions that are executed by the processor 301 to implement the various functions described above. The working memory 302 may include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, the storage unit 307 may include a hard disk drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc. The working memory 302 and the storage unit 307 may all be collectively referred to herein as memory or computer-readable storage media, and may be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by the processor 301 as a specific machine configured to implement the operations and functions described in the examples herein.

[0076] The input unit 306 may be any type of device capable of inputting information to the electronic device 300. The input unit 306 may receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and may include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone and / or a remote controller. The output unit may be any type of device capable of presenting information, and may include but is not limited to a display unit 305, a speaker 306, and other output units 309, which may include but are not limited to video / audio output terminals, vibrators and / or printers. The communication unit 308 allows the electronic device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, an 802.8 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0077] The application 302b in the working register 302 can be loaded to execute the various methods and processes described above, such as Figure 1. For example, in some embodiments, the method 100 described above may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 307. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the storage unit 307 and / or the communication unit 308. When the computer program is loaded and executed by the processor 301, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the processor 301 may be configured to execute the method 100 in any other appropriate manner (e.g., by means of firmware).

[0078] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0080] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0082] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0083] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be performed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document is not limited here.

[0085] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A video processing method, comprising: Acquire an input video, wherein the input video includes at least one input image frame and input audio; determining at least one qualified input frame having image content meeting an image quality standard from the at least one input image frame; Generate an output video, the output video comprising at least one output image frame corresponding to the qualified input frame, wherein the output audio of the output video is the same as the input audio, and the generating of the output video comprises: when the duration of the image sequence of the at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on the at least one output image frame, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

2. The video processing method according to claim 1, wherein: The at least one supplementary image frame is identical to a portion of the at least one output image frame.

3. The video processing method of claim 1, wherein converting the at least one output image frame and the at least one supplemental image frame into the output video comprises: reversing the order of the image sequence formed by the at least one output image frame to obtain a reversed image sequence, The at least one output image frame is merged with the image sequence in reverse order to obtain the output video.

4. The video processing method according to claim 1, further comprising: extracting an output image frame from the at least one generated output image frame; Determine whether the extracted output image frames meet the image quality standard to exclude unqualified output image frames.

5. The video processing method according to claim 1, wherein: The image quality criteria include at least one of the following: Whether the light in the image falls within the preset brightness range; Whether the shaking angle of the face and head in the image falls within the preset angle range; Whether the number of faces in the image falls within a preset range; Whether the ratio of the face size in the image is within a preset ratio range.

6. The video processing method according to any one of claims 1 to 5, wherein: When the number of the qualified input frames is less than a predetermined qualified threshold, the generation of the output video is abandoned.

7. The video processing method according to any one of claims 1 to 5, wherein: The at least one output image frame includes a predetermined avatar, and generating an output video including at least one output image frame corresponding to the qualified input frame includes: The at least one output image frame is generated according to the action of the character in the qualified input frame, wherein the virtual image in the at least one output image frame performs the action.

8. The video processing method according to claim 7, wherein: The actions include body movements and lip movements.

9. A video processing device, comprising: An input unit, configured to obtain an input video, wherein the input video includes at least one input image frame and input audio; an analyzing unit configured to determine at least one qualified input frame having image content meeting an image quality standard from the at least one input image frame; A generating unit is configured to generate an output video, the output video comprising at least one output image frame corresponding to the qualified input frame, wherein the output audio of the output video is the same as the input audio, and the generating of the output video comprises: when the duration of the image sequence of the at least one output image frame is less than the duration of the output audio, generating at least one supplementary image frame based on the at least one output image frame, and converting the at least one output image frame and the at least one supplementary image frame into the output video.

10. The video processing device according to claim 9, wherein: The at least one supplementary image frame is identical to a portion of the at least one output image frame.

11. The video processing device according to claim 9, wherein: Converting the at least one output image frame and the at least one supplemental image frame into the output video comprises: reversing the order of the image sequence formed by the at least one output image frame to obtain a reversed image sequence, The at least one output image frame is merged with the image sequence in reverse order to obtain the output video.

12. The video processing apparatus according to claim 9, generating the output video further comprising: extracting an output image frame from the at least one generated output image frame; Determine whether the extracted output image frames meet the image quality standard to exclude unqualified output image frames.

13. The video processing device according to claim 9, wherein: The image quality criteria include at least one of the following: Whether the light in the image falls within the preset brightness range; Whether the shaking angle of the face and head in the image falls within the preset angle range; Whether the number of faces in the image falls within a preset range; Whether the ratio of the face size in the image is within a preset ratio range.

14. The video processing device according to any one of claims 9 to 13, wherein: When the number of the qualified input frames is less than a predetermined qualified threshold, the generation of the output video is abandoned.

15. The video processing device according to any one of claims 9 to 13, wherein: The at least one output image frame includes a predetermined avatar, and generating an output video including at least one output image frame corresponding to the qualified input frame includes: The at least one output image frame is generated according to the action of the character in the qualified input frame, wherein the virtual image in the at least one output image frame performs the action.

16. The video processing device according to claim 15, wherein: The actions include body movements and lip movements.

17. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores a computer program, which, when executed by the at least one processor, implements the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing a computer program, wherein: The computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

19. A computer program product comprising a computer program, wherein: The computer program implements the method according to any one of claims 1 to 8 when executed by a processor.