Video processing method and device and electronic equipment

Abstract

The invention discloses a video processing method and device and electronic equipment. Relates to the field of data processing, and the method comprises the steps: obtaining an initial video which is a video of a broadcast text of a target object; determining an offset result of whether the target part of the target object has predetermined offset in the initial video; determining a virtual part corresponding to the target part under the condition that the deviation result is that the target part generates the preset deviation, and determining driving data corresponding to the virtual part according to the to-be-broadcasted text; and controlling the virtual part to cover the target part, and driving the virtual part to display by using the driving data to obtain a target video. According to the method and the device, the technical problem of poor watching user experience caused by difficulty in normal broadcasting in emergency when the target object broadcasts the text in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of video processing, and in particular to a video processing method, device and electronic equipment. Background Art

[0002] In the related technology, when a real person drives the display of a virtual image, a better virtual image display effect can be achieved, and the display of the virtual image can be made more vivid and flexible. However, when broadcasting long texts, the real person may encounter unexpected situations, which may cause the virtual image display effect to deteriorate during the broadcast, thereby affecting the user's viewing experience.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a video processing method, device and electronic device to at least solve the technical problem in the related art that when a target object broadcasts a text, it is difficult to broadcast normally when encountering an emergency, resulting in a poor viewing user experience.

[0005] According to one aspect of an embodiment of the present invention, there is provided a video processing method, comprising: obtaining an initial video, wherein the initial video is a video of a target object broadcasting text; determining in the initial video whether a target part of the target object undergoes a predetermined offset; when the offset result is that the target part undergoes the predetermined offset, determining a virtual part corresponding to the target part, and determining drive data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part to display with the drive data, to obtain a target video.

[0006] Optionally, the virtual part is controlled to cover the target part, and the virtual part is driven by the driving data to display, so as to obtain a target video, including: determining whether the target part restores the target position and obtaining a restoration result, wherein the target position is the position to which the current text being broadcast is adapted during the broadcast process; and when the restoration result is that the target part restores the target position, the virtual part is eliminated.

[0007] Optionally, after determining the offset result of whether a predetermined offset occurs in a target part of the target object in the initial video, the method further includes: when the offset result is that the target part occurs the predetermined offset, determining whether the offset of the target part affects the broadcast sound; when the broadcast impact result is that the broadcast sound is affected, determining virtual voice broadcast data based on the timbre data of the target object and the text to be broadcast; and controlling the initial video to be broadcast with the virtual voice broadcast data.

[0008] Optionally, the virtual part is controlled to cover the target part, and the virtual part is driven by the driving data to display, so as to obtain a target video, including: determining an alignment index between the virtual part and the target part in a frame without offset, wherein the frame without offset is a predetermined frame before the target part has the predetermined offset; when the alignment index is lower than a predetermined threshold, adjusting the shape parameters of the virtual part until the alignment index between the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold; controlling the adjusted virtual part to cover the target part in the frame without offset, and driving the virtual part to display with the driving data to obtain a target video.

[0009] Optionally, when the offset result is that the target part undergoes the predetermined offset, a virtual part corresponding to the target part is determined, and based on the text to be broadcast, driving data corresponding to the virtual part is determined, including: determining part parameters of the target part in the played video of the initial video; and determining the virtual part corresponding to the target part based on the part parameters.

[0010] Optionally, when the offset result is that the target part undergoes the predetermined offset, a virtual part corresponding to the target part is determined, and based on the text to be broadcast, driving data corresponding to the virtual part is determined, including: acquiring image parameters of the target object; and determining the virtual part corresponding to the target part based on the image parameters.

[0011] Optionally, the initial video is a live video, and the target object is a virtual object driven by a real person.

[0012] Optionally, after obtaining the initial video, it also includes: when a real person triggers a replacement instruction, determining a virtual part corresponding to the target part, and determining driving data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part with the driving data to display, so as to obtain the target video.

[0013] According to one aspect of an embodiment of the present invention, there is provided a video processing device, comprising: an acquisition module, for acquiring an initial video, wherein the initial video is a video of a target object broadcasting a text; a first determination module, for determining whether a target part of the target object undergoes a displacement result of a predetermined displacement in the initial video; a second determination module, for determining a virtual part corresponding to the target part when the displacement result is that the target part undergoes the predetermined displacement, and determining drive data corresponding to the virtual part based on the text to be broadcast; and a control module, for controlling the virtual part to cover the target part, and driving the virtual part to display with the drive data, so as to obtain a target video.

[0014] According to one aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the above-mentioned video processing methods.

[0015] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the above-mentioned video processing methods.

[0016] In an embodiment of the present invention, an initial video is obtained, wherein the initial video is a video of a target object broadcasting text. An offset result is determined as to whether a target part of the target object in the initial video has a predetermined offset. When the offset result is that a predetermined offset has occurred in the target part, a virtual part corresponding to the target part is determined, and drive data corresponding to the virtual part is determined based on the text to be broadcast. The virtual part is controlled to cover the target part, and the virtual part is driven by the drive data to display, so as to obtain a target video, so as to avoid part offset when the target object in the video broadcasts the text. The embodiment of the present invention achieves normal broadcasting and ensures the broadcasting quality by controlling the virtual part to cover the target part and driving it to display normally, thereby solving the technical problem in the related art that when the target object broadcasts the text, it is difficult to broadcast normally when encountering an emergency, resulting in a poor viewing user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flowchart of a video processing method according to an embodiment of the present invention;

[0019] Figure 2is a structural block diagram of a video processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0023] Example 1

[0024] According to an embodiment of the present invention, an embodiment of a video processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Figure 1 is a flow chart of a video processing method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0026] Step S102, obtaining an initial video, wherein the initial video is a video of the target object broadcasting the text;

[0027] In step S102 provided in the present application, an initial video is obtained, wherein the initial video is a video of the target object broadcasting the text.

[0028] The initial video can be a recorded video or a live video. By processing the recorded video, the final video can be presented without any part displacement. By processing the live video, the problem of unexpected situations during live broadcast can be avoided and normal playback can be ensured.

[0029] The target object can be a real person or a virtual person driven by a real person.

[0030] The initial video is a video of the target object broadcasting a text, that is, it can be a video of the target object broadcasting a text.

[0031] Step S104, determining whether a target part of the target object in the initial video has a predetermined offset result;

[0032] In step S104 provided in the present application, a shift result is determined as to whether a target part of a target object in an initial video has shifted by a predetermined amount.

[0033] That is, when the target object is broadcast according to the text, each part has a certain position at each stage of the broadcast. Therefore, it is necessary to determine whether the target part of the target object sends a predetermined offset to judge whether the target object has an unexpected situation such as leaving, falling, sneezing, etc., so as to make timely adjustments to the unexpected situation.

[0034] Step S106, when the offset result is that the target part has a predetermined offset, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined according to the text to be broadcasted;

[0035] In step S106 provided in the present application, the case where the offset result is that the target part is offset is described. In the case where the target object is offset, the offset virtual part is determined.

[0036] For example, when the target object leaves, all parts of the target object are offset, so the determined virtual part may be the entire virtual object.

[0037] For example, when the target object sneezes, it may be that the head of the target object is offset, and therefore, the determined virtual part may be the virtual head of the target object.

[0038] Etc., there is no limitation here, and it can be adaptively generated according to the actual offset condition.

[0039] In this step, the driving data corresponding to the virtual part is also generated according to the text to be broadcasted. Because in the process of initial video playback, the text before the offset is the text that has been broadcasted in the video, and the text after the offset is the text that has not been broadcasted, therefore, it is possible to determine how to correctly display the offset part according to the unbroadcasted text, and therefore, the driving data corresponding to the virtual part is generated for better display.

[0040] It should also be noted that when the virtual part is a virtual head, it is necessary to generate driving data corresponding to the virtual head, and the driving data corresponding to the virtual head includes driving data corresponding to the eyes, mouth, eyebrows, etc., so as to match the lip shape, eyebrow and eye movements of the unbroadcast text for a more vivid display.

[0041] Step S108, controlling the virtual part to cover the target part, and driving the virtual part to display with the driving data to obtain the target video.

[0042] In step S108 provided in the present application, the virtual part is controlled to cover the target part, and the virtual part is driven by driving data to display, so as to obtain the target video, so as to avoid unexpected conditions of the target object during video playback, resulting in poor playback effect.

[0043] Among them, the virtual part covers the target part, that is, the virtual part can replace the target part for correct display to avoid the above problems. It also allows the target part to be adaptively adjusted without the panic caused by still needing to be displayed on the screen.

[0044] Through the above steps S102-S108, an initial video is obtained, wherein the initial video is a video of the target object broadcasting the text. Determine in the initial video whether the target part of the target object has a predetermined offset. When the offset result is that the target part has a predetermined offset, determine the virtual part corresponding to the target part, and determine the drive data corresponding to the virtual part based on the text to be broadcast. Control the virtual part to cover the target part, and drive the virtual part with the drive data to display, so as to obtain the target video, so as to avoid the part offset when the target object in the video broadcasts the text. The embodiment of the present invention achieves the effect of normal broadcasting and ensuring the broadcasting quality by controlling the virtual part to cover the target part and driving it to display normally, thereby solving the technical problem in the related art that when the target object broadcasts the text, it is difficult to broadcast normally when encountering an emergency, resulting in a poor viewing user experience.

[0045] As an optional embodiment, the virtual part is controlled to cover the target part, and the driving data is used to drive the virtual part for display. After the target video is obtained, it also includes: determining whether the target part has restored the target position and obtaining a restoration result, wherein the target position is the position to which the current text being broadcast is adapted during the broadcast process; when the restoration result is that the target part has restored the target position, the virtual part is eliminated.

[0046] In this embodiment, the steps after obtaining the target video are described. Since the target part has been pre-determined offset, it is necessary to control the virtual part to cover it and replace it for broadcasting and displaying. At this time, the target part may be reset, for example, the broadcasting and displaying can be resumed after the unexpected situation is resolved. Therefore, it can be determined whether the target part has returned to the target position at this time.

[0047] It should be noted that what is determined here is whether the target part has restored the target position, which is the position corresponding to the current text being broadcast during the broadcast process. Because the virtual part has been broadcasting for a period of time instead of the target part, the position of the virtual part may change compared to when the broadcast just started. Therefore, it is necessary to determine whether the target position is where it should be now and determine the recovery result, so that when the recovery result is that the target part has restored to the target position, the virtual part is eliminated and the target part continues to be broadcast and displayed, making the broadcast more vivid and flexible.

[0048] As an optional embodiment, after determining in the initial video whether a target part of a target object has undergone a predetermined offset, the method further includes: when the offset result is that the target part has undergone a predetermined offset, determining whether the offset of the target part affects the broadcast sound; when the broadcast result is that the broadcast sound is affected, determining virtual voice broadcast data based on the timbre data of the target object and the text to be broadcast; and controlling the initial video to be broadcast with the virtual voice broadcast data.

[0049] In this embodiment, the situation after determining that the offset occurs is described. When the offset result is a predetermined offset of the target part, it is determined whether the offset of the target part affects the broadcast sound and the broadcast result. For example, if the target object walks away or sneezes, the offset of the target part will affect the broadcast sound. If the target object simply forgets the hand movement, the offset of the target part will not affect the broadcast sound. By judging whether the offset of the target part affects the broadcast sound and the broadcast result, targeted processing can be performed according to the impact on the broadcast result.

[0050] In the case where the broadcast result is affected by the broadcast sound, the virtual voice broadcast data is determined based on the timbre data of the target object and the text to be broadcast, so as to control the initial video to be broadcast with the virtual voice broadcast data, and obtain the target video with normal broadcast sound, so as to avoid the situation where the sound data cannot be broadcast normally due to accidents or emergencies.

[0051] It should be noted that in this process, attention should be paid to the matching degree between the virtual voice broadcast data and the virtual parts driven by the Quedong data, and attention should be paid to the synchronization of sound and picture to avoid the fragmentation effect caused by asynchrony.

[0052] As an optional embodiment, the virtual part is controlled to cover the target part, and the virtual part is driven by driving data for display to obtain a target video, including: determining an alignment index between the virtual part and the target part in a frame without offset, wherein the frame without offset is a predetermined frame before a predetermined offset occurs in the target part; when the alignment index is lower than a predetermined threshold, adjusting the shape parameters of the virtual part until the alignment index between the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold; controlling the adjusted virtual part to cover the target part in the frame without offset, and driving the virtual part for display with driving data to obtain a target video.

[0053] In this embodiment, the steps of controlling the virtual part to cover the target part and driving the virtual part to display with driving data to obtain the target video are described. The alignment index of the virtual part and the target part in the non-shifted frame is determined, wherein the non-shifted frame is a predetermined frame before the target part has a predetermined shift. For example, the previous frame.

[0054] Since the offset has already occurred when it is determined to have occurred, aligning the virtual part with the offset part will not make the broadcast proceed normally.

[0055] Therefore, it is necessary to align the virtual part with the part without offset to achieve the normal broadcasting effect.

[0056] Therefore, it is necessary to determine the alignment index between the two. When the alignment index is lower than a predetermined threshold, it means that the two are not aligned very well. At this time, it is necessary to adjust the shape parameters of the virtual part until the alignment index of the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold, that is, it meets the alignment standard.

[0057] The adjusted virtual part is controlled to cover the target part in the frame where no offset occurs, and the virtual part is driven by the driving data to display, so as to achieve the effect of obtaining the target video. In this way, the virtual part in the final video can be more perfectly matched and displayed. Moreover, for the user to watch, a freeze or pause of one frame or a predetermined frame will not be noticed, thus ensuring the user's viewing effect.

[0058] As an optional embodiment, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and based on the text to be broadcast, driving data corresponding to the virtual part is determined, including: determining the part parameters of the target part in the played video of the initial video; and determining the virtual part corresponding to the target part based on the part parameters.

[0059] In this embodiment, a specific method for determining a virtual part is described. The part parameters of the target part in the played video of the initial video are determined. That is, this embodiment determines the virtual part corresponding to the target part according to the part parameters of the target part in the played video. The determined virtual part is consistent with the target part played in the video, and the similarity index is extremely high.

[0060] As an optional embodiment, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined based on the text to be broadcast, including: obtaining image parameters of the target object; and determining the virtual part corresponding to the target part based on the image parameters.

[0061] In this embodiment, a specific method for determining virtual parts is described. The image parameters of the target object are obtained. That is, the image parameters of the target object can be stored in advance. According to the image parameters, each virtual part corresponding to the target object can be determined in advance, so that when an offset occurs, the virtual part corresponding to the target part can be directly retrieved, so as to achieve the effect of convenient and efficient retrieval of virtual parts.

[0062] As an optional embodiment, the initial video is a live video, and the target object is a virtual object driven by a real person.

[0063] In this embodiment, a scenario is described in which the initial video is a live video and the target object is a virtual object driven by a real person. When the initial video is a live video and the virtual object is driven by a real person, the method provided by the present application can avoid the problem of poor user viewing experience caused by unexpected situations during live broadcast. Moreover, since it is driven by a real person, unexpected situations are prone to occur. Since the real person drives the virtual object, the various virtual parts of the virtual object can also be determined quickly and efficiently. Furthermore, the recorded video can be repaired later, but the audience is watching in real time during the live broadcast, and it is impossible to perform later repairs. In this way, the problem of emergencies can be solved to a great extent.

[0064] As an optional embodiment, after obtaining the initial video, it also includes: when a real person triggers a replacement instruction, determining the virtual part corresponding to the target part, and determining the driving data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part with the driving data to display, so as to obtain the target video.

[0065] In this embodiment, the operations after obtaining the initial video are explained. In the scenario where the initial video is a live video and the target object is a virtual object driven by a real person, alternative instructions can also be triggered for the real person. For example, if the real person suddenly needs to leave, he or she can trigger the alternative instruction by himself or herself. The system thus responds to the alternative instruction, determines the virtual part corresponding to the target part, and determines the drive data corresponding to the virtual part based on the text to be broadcast; controls the virtual part to cover the target part, and drives the virtual part to display with the drive data, so as to obtain the target video, thereby achieving the purpose of spontaneously resolving sudden or unexpected situations.

[0066] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail below.

[0067] An optional implementation of the present invention provides a video processing method, which is described below:

[0068] S1, obtaining the target text to be broadcast;

[0069] S2, the real person broadcasts the target text normally in the predetermined area to drive the virtual person to broadcast and display in the video;

[0070] S3, when the real person leaves the predetermined area or the face disappears at the predetermined position, determining the action set, label set and voice set of the text-driven broadcast;

[0071] Generate target audio according to the timbre of a real person and the target text;

[0072] Acquire a motion change data set and an expression data set corresponding to the target virtual image, wherein the motion change data set includes motion change data corresponding to each sample phoneme in the sample speech;

[0073] S4, driving the target virtual image according to the action set, the voice set and the expression data set;

[0074] S5, when it is detected that the real person has returned, the real person broadcasting continues and the automatic broadcasting is cut off.

[0075] It also realizes the transition of movements, expressions and timbre between automatic broadcasting and real-person broadcasting.

[0076] Before making the transition, some preparation work can be done:

[0077] 1) Data collection and preprocessing: First, collect a large amount of data from automatic broadcasts and live broadcasts. This data should include audio, video, and text information. Preprocess the audio, including noise reduction, standardization, and feature extraction. For video data, extract key frames or features of actions and expressions.

[0078] 2) Feature extraction and modeling: Extract key features from the data of automatic and real-person broadcasts, such as timbre, intonation, speaking speed, facial expressions, and gestures. Use machine learning or deep learning algorithms to model these features so that intermediate transition states can be generated.

[0079] 3) Transition algorithm design: Design an algorithm to smoothly transition the features between automatic and live broadcasts. This can be achieved through interpolation, linear regression, dynamic time warping (DTW), or other time series analysis methods. The goal is to generate a natural transition effect so that users feel that the broadcast method is changing gradually.

[0080] 4) Synthesis and rendering: Use the extracted features and the designed transition algorithm to generate audio and video of the intermediate transition state. For audio, you can adjust parameters such as timbre, pitch, and speech speed. For video, you can synthesize key frames of actions and expressions to generate a smooth transition effect.

[0081] 5) Evaluation and optimization: Evaluate whether the generated transition effect is natural and smooth. The quality of the transition can be measured by manual evaluation or automatic evaluation methods. Based on the evaluation results, the algorithm and model are optimized to improve the quality of the transition effect.

[0082] Through the above optional implementation, at least the following beneficial effects can be achieved: the problem in the related art that when a real person drives a virtual person to display an image, the real person is prone to emergencies and it is difficult to display the image of the virtual person well is solved. The problem that the image of the virtual person can be well displayed even in emergencies is achieved.

[0083] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0084] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0085] Example 2

[0086] According to an embodiment of the present invention, a device for implementing the above video processing method is also provided. Figure 2 is a structural block diagram of a video processing device according to an embodiment of the present invention. Figure 2 As shown, the device includes: an acquisition module 202, a first determination module 204, a second determination module 206 and a control module 208. The device is described in detail below.

[0087] An acquisition module 202 is used to acquire an initial video, wherein the initial video is a video of the target object broadcasting the text;

[0088] A first determination module 204, connected to the acquisition module 202, is used to determine whether a target part of the target object in the initial video has a predetermined offset result;

[0089] The second determination module 206 is connected to the first determination module 204, and is used to determine the virtual part corresponding to the target part when the offset result is that the target part has a predetermined offset, and determine the driving data corresponding to the virtual part according to the text to be broadcasted;

[0090] The control module 208 is connected to the second determination module 206 and is used to control the virtual part to cover the target part, and drive the virtual part to display with the driving data to obtain the target video.

[0091] Among them, the control module 208 is also used to determine whether the target part restores the target position and obtain a restoration result, wherein the target position is the position to which the current text of the broadcast adapts during the broadcast process; when the restoration result is that the target part restores the target position, the virtual part is eliminated.

[0092] Among them, the first determination module 204 is also used to determine whether the target part of the target object in the initial video has a predetermined offset. After the offset result, it also includes: when the offset result is that the target part has a predetermined offset, determining whether the offset of the target part affects the broadcast sound; when the impact on the broadcast result is that it affects the broadcast sound, determining the virtual voice broadcast data based on the timbre data of the target object and the text to be broadcast; and controlling the initial video to be broadcast with the virtual voice broadcast data.

[0093] Among them, the control module 208 is also used to determine the alignment index between the virtual part and the target part in the frame without offset, wherein the frame without offset is a previous predetermined frame in which the target part has a predetermined offset; when the alignment index is lower than a predetermined threshold, adjust the shape parameters of the virtual part until the alignment index between the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold; control the adjusted virtual part to cover the target part in the frame without offset, and drive the virtual part to display with the driving data to obtain the target video.

[0094] The second determination module 206 is further used to determine the part parameters of the target part in the played video of the initial video; and determine the virtual part corresponding to the target part according to the part parameters.

[0095] The second determination module 206 is further used to obtain image parameters of the target object; and determine the virtual part corresponding to the target part according to the image parameters.

[0096] Optionally, the initial video is a live video, and the target object is a virtual object driven by a real person.

[0097] Among them, the acquisition module 202 is also used to determine the virtual part corresponding to the target part when a real person triggers the replacement instruction, and determine the driving data corresponding to the virtual part based on the text to be broadcast; control the virtual part to cover the target part, and drive the virtual part with the driving data to display, so as to obtain the target video.

[0098] It should be noted here that the above-mentioned acquisition module 202, first determination module 204, second determination module 206 and control module 208 correspond to steps S102 to S108 in implementing the video processing method, and the instances and application scenarios implemented by the multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0099] Example 3

[0100] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor, wherein the processor is configured to execute instructions to implement the following video processing method: obtaining an initial video, wherein the initial video is a video of a target object broadcasting text; determining in the initial video whether a target part of the target object has a predetermined offset; when the offset result is that the target part has a predetermined offset, determining a virtual part corresponding to the target part, and determining drive data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part to display with the drive data to obtain the target video.

[0101] Optionally, the virtual part is controlled to cover the target part, and the driving data is used to drive the virtual part for display to obtain a target video, including: determining whether the target part restores the target position and obtaining a restoration result, wherein the target position is the position to which the current text being broadcast is adapted during the broadcast process; when the restoration result is that the target part restores the target position, the virtual part is eliminated.

[0102] Optionally, after determining in the initial video whether a target part of the target object has undergone a predetermined offset, the method further includes: when the offset result is that the target part has undergone a predetermined offset, determining whether the offset of the target part affects the broadcast sound; when the broadcast result is that the broadcast sound is affected, determining virtual voice broadcast data based on the timbre data of the target object and the text to be broadcast; and controlling the initial video to be broadcast with the virtual voice broadcast data.

[0103] Optionally, the virtual part is controlled to cover the target part, and the virtual part is driven by driving data for display to obtain a target video, including: determining an alignment index between the virtual part and the target part in a frame without offset, wherein the frame without offset is a predetermined frame before a predetermined offset occurs in the target part; when the alignment index is lower than a predetermined threshold, adjusting a shape parameter of the virtual part until the alignment index between the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold; controlling the adjusted virtual part to cover the target part in the frame without offset, and driving the virtual part for display with driving data to obtain a target video.

[0104] Optionally, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and based on the text to be broadcast, driving data corresponding to the virtual part is determined, including: determining part parameters of the target part in the played video of the initial video; and determining the virtual part corresponding to the target part based on the part parameters.

[0105] Optionally, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined based on the text to be broadcast, including: obtaining image parameters of the target object; and determining the virtual part corresponding to the target part based on the image parameters.

[0106] Optionally, the initial video is a live video, and the target object is a virtual object driven by a real person.

[0107] Optionally, after obtaining the initial video, it also includes: when a real person triggers a replacement instruction, determining the virtual part corresponding to the target part, and determining the driving data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part with the driving data to display, so as to obtain the target video.

[0108] Example 4

[0109] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the following video processing method: obtaining an initial video, wherein the initial video is a video of a target object broadcasting text; determining in the initial video whether a target part of the target object undergoes a predetermined offset; when the offset result is that the target part undergoes a predetermined offset, determining a virtual part corresponding to the target part, and determining drive data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part to display with the drive data, to obtain the target video.

[0110] Optionally, the virtual part is controlled to cover the target part, and the driving data is used to drive the virtual part for display to obtain a target video, including: determining whether the target part restores the target position and obtaining a restoration result, wherein the target position is the position to which the current text being broadcast is adapted during the broadcast process; when the restoration result is that the target part restores the target position, the virtual part is eliminated.

[0111] Optionally, after determining in the initial video whether a target part of the target object has undergone a predetermined offset, the method further includes: when the offset result is that the target part has undergone a predetermined offset, determining whether the offset of the target part affects the broadcast sound; when the broadcast result is that the broadcast sound is affected, determining virtual voice broadcast data based on the timbre data of the target object and the text to be broadcast; and controlling the initial video to be broadcast with the virtual voice broadcast data.

[0112] Optionally, the virtual part is controlled to cover the target part, and the virtual part is driven by driving data for display to obtain a target video, including: determining an alignment index between the virtual part and the target part in a frame without offset, wherein the frame without offset is a predetermined frame before a predetermined offset occurs in the target part; when the alignment index is lower than a predetermined threshold, adjusting a shape parameter of the virtual part until the alignment index between the adjusted virtual part and the target part in the frame without offset is equal to or higher than the predetermined threshold; controlling the adjusted virtual part to cover the target part in the frame without offset, and driving the virtual part for display with driving data to obtain a target video.

[0113] Optionally, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and based on the text to be broadcast, driving data corresponding to the virtual part is determined, including: determining part parameters of the target part in the played video of the initial video; and determining the virtual part corresponding to the target part based on the part parameters.

[0114] Optionally, when the offset result is a predetermined offset of the target part, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined based on the text to be broadcast, including: obtaining image parameters of the target object; and determining the virtual part corresponding to the target part based on the image parameters.

[0115] Optionally, the initial video is a live video, and the target object is a virtual object driven by a real person.

[0116] Optionally, after obtaining the initial video, it also includes: when a real person triggers a replacement instruction, determining the virtual part corresponding to the target part, and determining the driving data corresponding to the virtual part based on the text to be broadcast; controlling the virtual part to cover the target part, and driving the virtual part with the driving data to display, so as to obtain the target video.

[0117] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0118] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: (steps of the method claim, sole right + subordinate right).

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0124] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0125] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0126] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0128] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A video processing method, characterized in that: include: Acquire an initial video, wherein the initial video is a video of the target object broadcasting the text; Determine whether a predetermined offset occurs in a target part of the target object in the initial video; In the case where the offset result is that the target part has the predetermined offset, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined according to the text to be broadcasted; The virtual part is controlled to cover the target part, and the virtual part is driven by the driving data to display, so as to obtain the target video.

2. The method according to claim 1, characterized in that controlling the virtual part to cover the target part, The virtual part is driven by the driving data to display, and a target video is obtained, including: Determine whether the target part has restored the target position, and obtain a restoration result, wherein the target position is the position to which the current text of the broadcast is adapted during the broadcast process; When the restoration result is that the target part has restored the target position, the virtual part is eliminated.

3. The method according to claim 1, characterized in that After determining whether a predetermined offset occurs to the target part of the target object in the initial video, the method further includes: In the case where the offset result is that the target part has the predetermined offset, determining whether the offset of the target part affects the broadcast sound and the broadcast result; In the case where the result of influencing the broadcast is that the broadcast sound is affected, determining virtual voice broadcast data according to the timbre data of the target object and the text to be broadcast; The initial video is controlled to be broadcasted using the virtual voice broadcasting data.

4. The method according to claim 1, characterized in that: controlling the virtual part to cover the target part, The virtual part is driven by the driving data to display, and a target video is obtained, including: Determining an alignment index between the virtual part and a target part in a frame where no offset occurs, wherein the frame where no offset occurs is a predetermined frame before the target part has the predetermined offset; When the alignment index is lower than a predetermined threshold, adjusting the shape parameter of the virtual part until the alignment index between the adjusted virtual part and the target part in the non-shifted frame is equal to or higher than the predetermined threshold; The adjusted virtual part is controlled to cover the target part in the frame where no offset occurs, and the virtual part is driven by the driving data to display, so as to obtain the target video.

5. The method according to claim 1, characterized in that In the case where the offset result is that the target part has the predetermined offset, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined according to the text to be broadcast, including: Determining a part parameter of the target part in a played video of the initial video; The virtual part corresponding to the target part is determined according to the part parameters.

6. The method according to claim 1, characterized in that In the case where the offset result is that the target part has the predetermined offset, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined according to the text to be broadcast, including: Acquire image parameters of the target object; The virtual part corresponding to the target part is determined according to the image parameter.

7. The method according to any one of claims 1 to 6, characterized in that: The initial video is a live video, and the target object is a virtual object driven by a real person.

8. The method according to claim 7, characterized in that After getting the initial video, also include: In the case where a real person triggers a replacement instruction, a virtual part corresponding to the target part is determined, and driving data corresponding to the virtual part is determined according to the text to be broadcasted; The virtual part is controlled to cover the target part, and the virtual part is driven by the driving data to display, so as to obtain the target video.

9. A video processing device, characterized in that: include: An acquisition module, used to acquire an initial video, wherein the initial video is a video of a target object broadcasting a text; A first determination module is used to determine whether a target part of the target object in the initial video has a predetermined offset result; A second determination module is used to determine a virtual part corresponding to the target part when the offset result is that the target part has the predetermined offset, and determine driving data corresponding to the virtual part according to the text to be broadcasted; The control module is used to control the virtual part to cover the target part, and drive the virtual part to display with the driving data to obtain the target video.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video processing method according to any one of claims 1 to 8.

Images