A video playing method and device, electronic equipment and computer readable medium

Abstract

The application discloses a video playing method and device, electronic equipment and computer readable medium, and relates to the technical field of computers. A specific embodiment comprises obtaining script data, close-up video and voice-over audio according to a received video playing request; inserting the voice-over audio into the script data based on a voice-over label in the script data to generate target script data; splitting the target script data according to the voice-over audio and a preset character quantity threshold to generate segmented target script data; determining the state of a virtual image used based on the data type of the segmented target script data; in response to the segmented target script data containing a shot label, generating a target segmented video based on the shot label, the state and the segmented target script data; and generating a live broadcast task according to each target segmented video and executing the live broadcast task to complete the playing of a corresponding video. The application improves the efficiency and flexibility of virtual image video playing and enhances the video playing effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video playback method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] Currently, digital human videos are created separately from text and audio / video clips, which are then played sequentially. This separate creation of text and audio / video increases script complexity and makes it difficult to visually grasp the complete script. The script layout is fixed, and any adjustments to camera positions require re-creation. Inserted videos or audio can only be played after the entire digital human script video has been played, resulting in low playback efficiency, low flexibility, and poor visual quality. Summary of the Invention

[0003] In view of this, embodiments of this application provide a video playback method, apparatus, electronic device, and computer-readable medium, which can solve the problems of low efficiency, low flexibility, and poor effect of existing digital human video playback.

[0004] To achieve the above objectives, according to one aspect of the embodiments of this application, a video playback method is provided, comprising:

[0005] Retrieve script data, close-up video, and voice-over audio based on the received video playback request;

[0006] Based on the voice-over tags in the script data, the voice-over audio is inserted into the script data to generate the target script data;

[0007] Based on the voice-over audio and a preset character count threshold, the target script data is split to generate segmented target script data;

[0008] Based on the data type of the segmented target script data, determine the state of the virtual avatar used, and in response to including shot tags in the segmented target script data, generate the target segmented video based on the shot tags, state, and segmented target script data;

[0009] Based on the video segments of each target, a broadcast task is generated and executed to complete the playback of the corresponding video.

[0010] Optionally, the target segmented video is generated, including:

[0011] Based on the segmented target script data and status, generate corresponding segmented videos;

[0012] Determine the time point at which the shot label is read aloud in the segmented video, and insert the close-up video corresponding to the shot label into the segmented video at the time point to generate the target segmented video.

[0013] Optionally, the target script data is split based on the voice-over audio and a preset character count threshold to generate segmented target script data, including:

[0014] The target script data is split from the location of the voice-over audio to obtain individual split data;

[0015] If any of the split data contains more than a preset character count threshold, the split data exceeding the preset character count threshold will be split again according to the preset character count threshold to generate segmented target script data.

[0016] Optionally, the split data exceeding the preset character count threshold can be further split according to the preset character count threshold, including:

[0017] Identify punctuation marks at positions where the number of characters is less than or equal to the preset character count threshold in split data that exceeds the preset character count threshold.

[0018] Split data at punctuation marks where the number of characters exceeds a preset threshold.

[0019] Optionally, based on the data type of the segmented target script data, the state of the virtual avatar used is determined, including:

[0020] The virtual avatar used for voice-over audio data in the segmented target script data is defined as the breathing state.

[0021] The virtual avatars used for text-based data in the segmented target script data are defined as being in a non-breathing state.

[0022] Optionally, before generating the broadcast task, the method further includes:

[0023] Arrange the target video segments according to the script data to generate a complete video;

[0024] Read the launch script data, compare the launch script data with the complete video, and if the comparison result is consistent, execute the launch task generation process.

[0025] Optionally, before generating the broadcast task, the method further includes:

[0026] Read the script data for the live broadcast, compare the script data with the complete video, and determine the type of inconsistency in response to a discrepancy in the comparison result.

[0027] If the position of the close-up video is inconsistent, recalculate the position of the close-up video and fill it back into the corresponding recalculated position;

[0028] If the video type is not a close-up shot and the video position is inconsistent, execute the re-production process.

[0029] In addition, this application also provides a video playback device, including:

[0030] The acquisition unit is configured to acquire script data, close-up video, and voice-over audio based on a received video playback request;

[0031] The target script data generation unit is configured to insert voice-over audio into the script data based on the voice-over tags in the script data to generate target script data.

[0032] The splitting unit is configured to split the target script data based on the voice-over audio and a preset character count threshold to generate segmented target script data;

[0033] The target segmented video generation unit is configured to determine the state of the virtual avatar used based on the data type of the segmented target script data, and in response to the inclusion of shot tags in the segmented target script data, generate the target segmented video based on the shot tags, state, and segmented target script data.

[0034] The video playback unit is configured to generate a playback task based on each target video segment, and execute the playback task to complete the playback of the corresponding video.

[0035] Optionally, the target segmented video generation unit is further configured to:

[0036] Based on the segmented target script data and status, generate corresponding segmented videos;

[0037] Determine the time point at which the shot label is read aloud in the segmented video, and insert the close-up video corresponding to the shot label into the segmented video at the time point to generate the target segmented video.

[0038] Optionally, the split unit is further configured as follows:

[0039] The target script data is split from the location of the voice-over audio to obtain individual split data;

[0040] If any of the split data contains more than a preset character count threshold, the split data exceeding the preset character count threshold will be split again according to the preset character count threshold to generate segmented target script data.

[0041] Optionally, the split unit is further configured as follows:

[0042] Identify punctuation marks at positions where the number of characters is less than or equal to the preset character count threshold in split data that exceeds the preset character count threshold.

[0043] Split data at punctuation marks where the number of characters exceeds a preset threshold.

[0044] Optionally, the target segmented video generation unit is further configured to:

[0045] The virtual avatar used for voice-over audio data in the segmented target script data is defined as the breathing state.

[0046] The virtual avatars used for text-based data in the segmented target script data are defined as being in a non-breathing state.

[0047] Optionally, the video playback unit is further configured to:

[0048] Arrange the target video segments according to the script data to generate a complete video;

[0049] Read the launch script data, compare the launch script data with the complete video, and if the comparison result is consistent, execute the launch task generation process.

[0050] Optionally, the video playback unit is further configured to:

[0051] Read the script data for the live broadcast, compare the script data with the complete video, and determine the type of inconsistency in response to a discrepancy in the comparison result.

[0052] If the position of the close-up video is inconsistent, recalculate the position of the close-up video and fill it back into the corresponding recalculated position;

[0053] If the video type is not a close-up shot and the video position is inconsistent, execute the re-production process.

[0054] In addition, this application also provides a video playback electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the video playback method described above.

[0055] In addition, this application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the video playback method described above.

[0056] To achieve the above objectives, according to another aspect of the embodiments of this application, a computer program product is provided.

[0057] A computer program product according to an embodiment of this application includes a computer program that, when executed by a processor, implements the video playback method provided in an embodiment of this application.

[0058] One embodiment of the above invention has the following advantages or beneficial effects: This application obtains script data, close-up video, and voice-over audio based on a received video playback request; inserts the voice-over audio into the script data based on the voice-over tags in the script data to generate target script data; splits the target script data according to the voice-over audio and a preset character count threshold to generate segmented target script data; determines the state of the virtual avatar used based on the data type of the segmented target script data; responds by including shot tags in the segmented target script data; and generates target segmented videos based on the shot tags, state, and segmented target script data; generates a broadcast task based on each target segmented video and executes the broadcast task to complete the playback of the corresponding video. This improves the efficiency and flexibility of virtual avatar (e.g., digital human) video playback and enhances the video playback effect.

[0059] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0060] The accompanying drawings are provided to better understand this application and do not constitute an undue limitation thereof. Wherein:

[0061] Figure 1 This is a schematic diagram illustrating the main flow of a video playback method according to an embodiment of this application;

[0062] Figure 2 This is a schematic diagram illustrating the main flow of a video playback method according to an embodiment of this application;

[0063] Figure 3 This is a schematic flowchart of a video playback method according to an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of the script splitting and production process of a video playback method according to an embodiment of this application;

[0065] Figure 5 This is a schematic diagram of the video playback method according to an embodiment of this application;

[0066] Figure 6 This is a schematic diagram of a dataset obtained by integrating video according to a video playback method of one embodiment of this application;

[0067] Figure 7 This is a schematic diagram of the main units of a video playback device according to an embodiment of this application;

[0068] Figure 8 This is an exemplary system architecture diagram to which embodiments of this application can be applied;

[0069] Figure 9This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers in the embodiments of this application. Detailed Implementation

[0070] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application comply with relevant national laws and regulations. It should also be noted that certain software, components, models, and other existing industry solutions may be mentioned in the embodiments of this application. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions. The collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions of this application all comply with relevant laws and regulations, are used for legal and reasonable purposes, do not violate public order and good morals, are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to user personal information, safeguard user personal information security, cybersecurity, and national security, and ensure that those authorized to access personal information comply with relevant laws and regulations. Once this user personal information is no longer needed, risks should be minimized by restricting or even prohibiting data collection and / or deleting the data.

[0071] When used, including in certain relevant applications, data is deidentified to protect user privacy, for example by removing specific identifiers, controlling the amount or specificity of stored data, controlling how data is stored, and / or other methods.

[0072] Figure 1 This is a schematic diagram illustrating the main flow of a video playback method according to an embodiment of this application, as shown below. Figure 1 As shown, the video playback method mainly includes the following steps S101-S105.

[0073] Step S101: Obtain script data, close-up video, and voice-over audio based on the received video playback request.

[0074] In this embodiment, the execution entity of the video playback method (e.g., a server) can receive video playback requests via wired or wireless connections. Upon receiving a video playback request, the execution entity can obtain the script data corresponding to the request, such as: "Silent | Shot 1, Sound | Shot 2, Cut, Sound | Shot 2 Next, let me introduce the ding~ bell sound.mp3 item at a very good price. In addition, you can enjoy discounts when ordering in the live stream! Those who are still hesitating can place their orders directly," close-up video, and voice-over audio.

[0075] Step S102: Based on the voice-over tags in the script data, insert the voice-over audio into the script data to generate the target script data.

[0076] Voice-over tags, such as "ding~bell sound.mp3". The voice-over audio is inserted into the corresponding positions of the voice-over tags to generate target script data, which contains the script text and the voice-over audio.

[0077] Step S103: Based on the voice-over audio and a preset character count threshold, split the target script data to generate segmented target script data.

[0078] Specifically, based on the voice-over audio and a preset character count threshold, the target script data is split to generate segmented target script data, including: splitting the target script data from the position of the voice-over audio to obtain each segmented data; if there is a segmented data in each segmented data that is greater than the preset character count threshold, the segmented data that is greater than the preset character count threshold is split again according to the preset character count threshold, thereby generating segmented target script data.

[0079] Specifically, the split data that exceeds the preset character count threshold is split again according to the preset character count threshold, including: identifying punctuation marks in the split data that exceeds the preset character count threshold at positions less than or equal to the preset character count threshold; and splitting the split data that exceeds the preset character count threshold from the punctuation marks.

[0080] For example, script splitting: the script is split from the position with off-screen audio. If the script is still longer than the production limit after splitting, it is split again according to the preset character count threshold. The splitting rule is to find the first punctuation mark that is shorter than the production limit and truncate it. This way, the segmented target script data that meets the production length of the algorithm model can be obtained accurately and quickly, improving the efficiency and accuracy of video playback.

[0081] Step S104: Based on the data type of the segmented target script data, determine the state of the virtual avatar used, and in response to including shot tags in the segmented target script data, generate the target segmented video based on the shot tags, state, and segmented target script data.

[0082] Specifically, based on the data type of the segmented target script data, the state of the virtual avatar used is determined, including: determining the state of the virtual avatar used for data of the data type of voice-over audio in the segmented target script data as breathing state; and determining the state of the virtual avatar used for data of the data type of text in the segmented target script data as non-breathing state.

[0083] Voice-over audio is created using a breathing virtual avatar (e.g., a digital human) (mouth closed, no body movement), while text is created using a non-breathing virtual avatar (able to speak, with body movement). By inserting the corresponding close-up video into the positions corresponding to the shot tags, target segmented videos can be generated efficiently and accurately based on the determined state (breathing or non-breathing) of the virtual avatar (e.g., a digital human) and segmented target script data.

[0084] Specifically, the corresponding video can be generated first based on the determined state of the virtual character and the segmented target script data. Then, the corresponding close-up video can be inserted at the position corresponding to the shot label in the corresponding video to quickly and accurately obtain the target segmented video.

[0085] Step S105: Generate a broadcast task based on each target video segment, and execute the broadcast task to complete the playback of the corresponding video.

[0086] Specifically, before generating the launch task, the method also includes: arranging each target segment video according to the script data to generate a complete video; reading the launch script data, comparing the launch script data with the complete video, and executing the launch task generation process in response to the comparison result being consistent.

[0087] The launch is the process of arranging and playing all text-based videos, voice-over videos, and close-up videos according to the configured script. It is necessary to realign the final product (the complete video generated by arranging the target video segments according to the script data) with the original script (e.g., the script closest to the final video's timing before the actual video's timing) to ensure the launch effect meets the user's expectations.

[0088] Specifically, before generating the launch task, the method also includes: reading the launch script data, comparing the launch script data with the complete video, and in response to an inconsistency in the comparison result, determining the type of inconsistency; if the type is an inconsistency in the position of the close-up video, recalculating the position of the close-up video and filling the close-up video back into the recalculated corresponding position; if the type is not an inconsistency in the position of the close-up video, executing the re-production process.

[0089] The only type of inconsistency between the script data and the complete video is when the close-up video position is inconsistent. This is because any modification to the script will lead to re-production, which consumes a lot of resources. The shot itself is video and does not need to be re-produced. Adjusting the shot position has no impact on the script video itself. Therefore, when the user modifies the shot position, it will not trigger re-production. Instead, the close-up video will be recalculated and put back into the corresponding position when the broadcast starts.

[0090] This embodiment obtains script data, close-up video, and voice-over audio based on a received video playback request; inserts the voice-over audio into the script data based on the voice-over tags in the script data to generate target script data; splits the target script data according to the voice-over audio and a preset character count threshold to generate segmented target script data; determines the state of the virtual avatar used based on the data type of the segmented target script data, and in response to including shot tags in the segmented target script data, generates target segmented videos based on the shot tags, state, and segmented target script data; generates a broadcast task based on each target segmented video, and executes the broadcast task to complete the playback of the corresponding video. This improves the efficiency and flexibility of virtual avatar (e.g., digital human) video playback, and enhances the video playback effect.

[0091] Figure 2 This is a schematic diagram of the main flow of a video playback method according to an embodiment of this application, as shown below. Figure 2 As shown, the video playback method mainly includes the following steps S201-S207.

[0092] Step S201: Obtain script data, close-up video, and voice-over audio based on the received video playback request.

[0093] The script data obtained from the received video playback request may contain only text data, which may include various tags to represent the voice-over audio or close-up video to be inserted into the script data. The executing entity can also obtain the uploaded close-up video and voice-over audio corresponding to the video playback request.

[0094] Step S202: Based on the voice-over tags in the script data, insert the voice-over audio into the script data to generate the target script data.

[0095] The executing entity can identify voice-over tags in the script data. For example, tags containing .mp3 are voice-over tags. The voice-over audio corresponding to the voice-over tag is inserted into the corresponding voice-over tag in the script data to generate the target script data.

[0096] Step S203: Based on the voice-over audio and a preset character count threshold, split the target script data to generate segmented target script data.

[0097] Step S204: Determine the status of the virtual avatar to be used based on the data type of the segmented target script data.

[0098] The data types of segmented target script data can include voice-over audio and text. When creating a video, the virtual avatar used for voice-over audio is in a breathing state, while the virtual avatar used for text is in a non-breathing state. Specifically, the breathing state means the virtual avatar (i.e., the digital human) is in a closed-mouth, non-moving state. The non-breathing state means the virtual avatar (i.e., the digital human) is in a speaking, moving-body state.

[0099] Step S205: In response to including shot tags in the segmented target script data, generate corresponding segmented videos based on the segmented target script data and status.

[0100] If the segmented target script data contains shot tags, the system will first call the corresponding algorithm model based on the segmented target script data and the state of the corresponding virtual character to quickly and accurately generate the corresponding segmented video.

[0101] Step S206: Determine the time point when the shot label is read aloud in the segmented video, and insert the close-up video corresponding to the shot label into the segmented video at the time point to generate the target segmented video.

[0102] While playing segmented videos, a virtual avatar is invoked to activate the voice-to-text function. The avatar reads the text corresponding to the target script data for each segment of the generated video at a preset speaking speed. The system records the time points in the segmented videos when the camera tags corresponding to the target script data are read, and inserts close-up video clips corresponding to those camera tags at those time points. This process accurately and efficiently generates the target segmented videos, improving video playback quality, efficiency, and flexibility.

[0103] Step S207: Generate a broadcast task based on each target video segment, and execute the broadcast task to complete the playback of the corresponding video.

[0104] The executing entity can send the broadcast task to the client, so that the client can play the corresponding video according to the broadcast task.

[0105] In this embodiment, the script is a text that the digital human reads aloud according to a specified image, tone, and speaking speed. Production involves using an algorithmic model to create a digital human video from the script. Silent shot: A video inserted while the digital human is playing; the shot covers the digital human's image, but the digital human's voice is not interrupted. Speech shot: A video inserted while the digital human is playing; the shot covers the digital human's image, and the digital human's voice is interrupted, playing the video's own audio. Digital human breathing state: A digital human state where the image is generally closed and without physical movement. Off-screen audio: Audio inserted while the digital human is playing, interrupting the digital human's voice; the digital human is in a breathing state during the audio playback. This is often used to create a live stream atmosphere, such as: "123, link up!" or "Link up!"

[0106] Figure 3 This is a schematic diagram of the overall flow of a video playback method according to an embodiment of this application. Figure 3 As shown, the script can be created, written, and audio inserted. The script can be split based on the audio and then the text script and audio can be produced based on the split script to obtain digital human video clips. In addition, the relative position of the shot video can be calculated, and the shot video can be inserted at the corresponding position in the digital human video clip before it is played.

[0107] For example, the script is as follows:

[0108] Silent | Scene 1, Sound | Scene 2, Continue, Sound | Scene 2 Next, let me introduce the "Ding~ Bell Sound.mp3" item, which is very affordable. In addition, you'll enjoy further discounts when ordering during the live stream! Those who are still hesitating can place their orders now!

[0109] In this script, "Silent | Shot 1" and "Shot with Sound | Shot 2" are video clips, and "Ding~ Bell sound.mp3" is the audio voice-over. Video clips and audio voice-overs can be labeled with different colors in the script. For example, blue label: audio voice-over, red label: video clips.

[0110] Figure 4 This is a schematic diagram of the script splitting and production process of a video playback method according to an embodiment of this application. Figure 4As shown, the process begins by filtering the video footage in the script to obtain a list of video footage. It then determines if there is any voice-over audio. If so, the script is truncated based on the voice-over audio, and the script length is checked against a preset character threshold. If there is no voice-over audio, the script length is checked against the preset character threshold again. If the script length is less than the preset character threshold, a production task is generated based on the video footage list. For voice-over tasks in the production task, a corresponding video is created using a breathing state; for tasks that are not voice-over audio, a corresponding video is created using a non-breathing state. The video production then ends. If the script length is greater than the preset character threshold, the script is truncated according to the preset character threshold, and a production task is generated based on the video footage list. For voice-over tasks in the production task, a corresponding video is created using a breathing state; for tasks that are not voice-over audio, a corresponding video is created using a non-breathing state. The video production then ends.

[0111] Figure 5 This is a schematic diagram illustrating the video playback process according to an embodiment of this application. Figure 5 As shown, the process begins by reading the script for the live broadcast and determining if the script has been modified (excluding the video footage). If so, the video is remade. Otherwise, the production task list is read, and it is determined whether it contains video footage. If it does, a list of video footage is obtained, the relative positions of the video footage are calculated, and these calculated relative positions are then filled into the corresponding pre-made videos. The live broadcast begins, and the process ends. If the production task list does not contain video footage, the live broadcast task is generated directly, the live broadcast begins, and the process ends.

[0112] As an example of the video playback method in this application, the specific implementation process of video playback can be as follows:

[0113] Upload resources: The system provides a close-up lens library, where users can upload their shots; and it also provides a voice-over library, where users can upload their audio.

[0114] like Figure 3 As shown, the process involves creating a script, writing the script, and inserting shots. After the user has written the script, they can insert video shots and voice-over audio at appropriate locations. Script splitting: The algorithm model has a limited length for each script produced at once; excessively long scripts need to be split into multiple segments. Voice-over audio is produced using breathing virtual avatars (e.g., digital humans) (closed mouth, no body movements), while text is produced using non-breathing virtual avatars.

[0115] Filtering shots in the script: Shots in the script do not need to be produced, but it is necessary to record which part of the script the shot belongs to, and insert the shot into the corresponding position when the broadcast begins.

[0116] Script splitting: Split the script at the position with voice-over audio. If the script is still longer than the production limit after splitting, split it again according to the preset character count threshold. The splitting rule is to find the first punctuation mark that is shorter than the production limit and truncate it.

[0117] Generate production tasks: Based on the script breakdown, submit each voice-over segment and script to the algorithm model to produce a video.

[0118] Launch: Launching is the process of arranging and playing all text-based videos, voice-over videos, and camera footage according to the configured script. It's necessary to re-align the final product with the original script (e.g., the script closest to the original video's timing before the actual video's start time) to ensure the launch effect meets user expectations.

[0119] Determining if reproduction is needed: When the program starts airing, the script is read, and the production task and script are compared to determine if the script has been modified after production. If it has been modified, it needs to be reproduced.

[0120] Shot Video Refilling: Because any modification to the script will lead to re-production, which consumes significant resources, while shots are inherently video and require no re-production, adjusting shot positions has no impact on the script video itself. Therefore, user modifications to shot positions will not trigger re-production. Instead, close-up video will be re-filled into the corresponding positions during broadcast by recalculation. The production task and corresponding script are read, and it is determined whether the text contains close-up video. The close-up video is extracted, and a mark is left at the location of the close-up video. The TTS service is called to obtain the time when the voice reads the text to the marked position and this time is recorded in the broadcast task.

[0121] The broadcast will integrate the script video, voice-over video, and close-up video into a single format. Figure 6 The structure shown corresponds to such a dataset for each script. The data is sent to the client, and playback is performed according to the rules. (Example) Figure 6 The integrated dataset can be seen as follows: Script: Effectively protects eye health and relieves eye fatigue || Voiceover: Ding~ bell sound || Shot: Sound | Shot 2 || Voiceover: Price is set, refresh and shoot || Script: Very suitable... You can enjoy a super exclusive price. || Shot: Sound | Shot 2 || Script: And it can also...

[0122] In the integrated dataset, "||" represents the segmentation interval symbol, which can represent... Figure 6 The integrated video includes video transition nodes, such as transition nodes from script video to voice-over video, and transition nodes from voice-over video to close-up video.

[0123] This application embodiment uses a mix of text and tags to format the script and any inserted audio and video, creating a digital human video for playback. Without altering the text script, adjusting the video position allows for playback without re-creation. Audio and video can be inserted and played at any position within the script.

[0124] Figure 7 This is a schematic diagram of the main units of a video playback device according to an embodiment of this application. Figure 7 As shown, the video playback device 700 includes an acquisition unit 701, a target script data generation unit 702, a splitting unit 703, a target segmented video generation unit 704, and a video playback unit 705.

[0125] Acquisition unit 701 is configured to acquire script data, close-up video, and voice-over audio based on a received video playback request.

[0126] The target script data generation unit 702 is configured to insert voice-over audio into the script data based on the voice-over tags in the script data to generate target script data.

[0127] The splitting unit 703 is configured to split the target script data based on the voice-over audio and a preset character count threshold to generate segmented target script data.

[0128] The target segmented video generation unit 704 is configured to determine the state of the virtual avatar used based on the data type of the segmented target script data, and in response to including shot tags in the segmented target script data, generate the target segmented video based on the shot tags, state, and segmented target script data.

[0129] The video playback unit 705 is configured to generate a playback task based on each target video segment and execute the playback task to complete the playback of the corresponding video.

[0130] In some embodiments, the target segmented video generation unit 704 is further configured to: generate corresponding segmented videos based on segmented target script data and status; determine the time point at which the shot tag is read aloud in the segmented video; and insert close-up video corresponding to the shot tag at the time point in the segmented video to generate the target segmented video.

[0131] In some embodiments, the splitting unit 703 is further configured to: split the target script data from the position of the voice-over audio to obtain each split data; if there is a split data in each split data that is greater than a preset character number threshold, split the split data that is greater than the preset character number threshold again according to the preset character number threshold, thereby generating segmented target script data.

[0132] In some embodiments, the splitting unit 703 is further configured to: identify punctuation marks at positions less than or equal to the preset character count threshold in the split data that is greater than the preset character count threshold; and split the split data that is greater than the preset character count threshold from the punctuation marks.

[0133] In some embodiments, the target segmented video generation unit 704 is further configured to: determine the state of the virtual image used for data of data with data type of voice-over audio in the segmented target script data as a breathing state; and determine the state of the virtual image used for data of data with data type of text in the segmented target script data as a non-breathing state.

[0134] In some embodiments, the video playback unit 705 is further configured to: arrange the various target segmented videos according to script data to generate a complete video; read the launch script data, compare the launch script data with the complete video, and execute the launch task generation process in response to the comparison result being consistent.

[0135] In some embodiments, the video playback unit 705 is further configured to: read the opening script data, compare the opening script data with the complete video, and in response to a discrepancy in the comparison result, determine the type of discrepancy; if the type is a discrepancy in the position of the close-up video, recalculate the position of the close-up video and fill the close-up video back into the recalculated corresponding position; if the type is not a discrepancy in the position of the close-up video, perform a re-production process.

[0136] It should be noted that the video playback method and video playback device in this application are related in terms of specific implementation content, so repeated content will not be described again.

[0137] Figure 8 An exemplary system architecture 800 is shown that can be applied to the video playback method or video playback device of the embodiments of this application.

[0138] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, and 803, a network 804, and a server 805. Network 804 serves as the medium for providing communication links between terminal devices 801, 802, and 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0139] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0140] Terminal devices 801, 802, and 803 can be various electronic devices with video playback and processing screens that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0141] Server 805 can be a server providing various services, such as a backend management server supporting video playback requests submitted by users using terminal devices 801, 802, and 803 (for example only). The backend management server can obtain script data, close-up video, and voice-over audio based on the received video playback request; insert the voice-over audio into the script data based on the voice-over tags in the script data to generate target script data; split the target script data according to the voice-over audio and a preset character count threshold to generate segmented target script data; determine the state of the virtual avatar used based on the data type of the segmented target script data; respond by including shot tags in the segmented target script data; and generate target segmented videos based on the shot tags, state, and segmented target script data; generate playback tasks based on each target segmented video and execute the playback tasks to complete the playback of the corresponding videos. This improves the efficiency and flexibility of virtual avatar (e.g., digital human) video playback and enhances the video playback effect.

[0142] It should be noted that the video playback method provided in this application embodiment is generally executed by server 805, and correspondingly, the video playback device is generally set in server 805.

[0143] It should be understood that Figure 8 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0144] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 900 suitable for implementing a terminal device according to the embodiments of this application. Figure 9 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0145] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the computer system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0146] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0147] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of this application.

[0148] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a target script data generation unit, a splitting unit, a target segmented video generation unit, and a video playback unit. The names of these units do not necessarily limit the specific unit itself.

[0151] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to obtain script data, close-up video, and voice-over audio according to a received video playback request; insert the voice-over audio into the script data based on the voice-over tags in the script data to generate target script data; split the target script data according to the voice-over audio and a preset character count threshold to generate segmented target script data; determine the state of the virtual avatar used based on the data type of the segmented target script data; in response to including shot tags in the segmented target script data, generate target segmented videos based on the shot tags, state, and segmented target script data; generate a broadcast task according to each target segmented video, and execute the broadcast task to complete the playback of the corresponding video.

[0152] The computer program product of this application includes a computer program that, when executed by a processor, implements the video playback method in the embodiments of this application.

[0153] According to the technical solutions of the embodiments of this application, the efficiency and flexibility of video playback of virtual avatars (such as digital humans) can be improved, as well as the video playback effect can be enhanced.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A video playback method, characterized in that, include: Retrieve script data, close-up video, and voice-over audio based on the received video playback request; Based on the voice-over tags in the script data, the voice-over audio is inserted into the script data to generate the target script data; Based on the off-screen audio and a preset character count threshold, the target script data is split to generate segmented target script data; Based on the data type of the segmented target script data, the state of the virtual avatar used is determined, and in response to the inclusion of shot tags in the segmented target script data, a target segmented video is generated based on the shot tags, the state, and the segmented target script data. Based on each of the target video segments, a broadcast task is generated, and the broadcast task is executed to complete the playback of the corresponding video.

2. The method according to claim 1, characterized in that, The generation of the target segmented video includes: Based on the segmented target script data and the status, generate corresponding segmented videos; Determine the time point at which the shot tag is read aloud in the segmented video, and insert the close-up video corresponding to the shot tag into the segmented video at the time point to generate the target segmented video.

3. The method according to claim 1, characterized in that, The step of splitting the target script data based on the off-screen audio and a preset character count threshold to generate segmented target script data includes: The target script data is split from the location of the voice-over audio to obtain individual split data. If any of the split data contains a number of characters greater than a preset character count threshold, the split data containing the number of characters greater than the preset character count threshold is split again according to the preset character count threshold to generate segmented target script data.

4. The method according to claim 3, characterized in that, The step of further splitting the data that exceeds the preset character count threshold according to the preset character count threshold includes: Identify punctuation marks at positions in the split data that are greater than or equal to the preset character count threshold; The data that exceeds a preset character count threshold is split from the punctuation marks.

5. The method according to claim 1, characterized in that, The determination of the status of the virtual avatar used based on the data type of the segmented target script data includes: The virtual avatar used for voice-over audio data in the segmented target script data is defined as a breathing state. The virtual avatar used for text-based data in the segmented target script data is determined to be in a non-breathing state.

6. The method according to claim 1, characterized in that, Prior to generating the broadcast task, the method further includes: The target video segments are arranged according to the script data to generate a complete video; Read the live broadcast script data, compare the live broadcast script data with the complete video, and if the comparison result is consistent, execute the live broadcast task generation process.

7. The method according to claim 6, characterized in that, Prior to generating the broadcast task, the method further includes: Read the script data for the start of the broadcast, compare the script data with the complete video, and determine the type of inconsistency in response to a discrepancy in the comparison result. If the type of video is a close-up shot with inconsistent positions, recalculate the position of the close-up shot video and fill it back into the recalculated position. If the video position is inconsistent and the type is not a close-up shot, a re-production process will be executed.

8. A video playback device, characterized in that, include: The acquisition unit is configured to acquire script data, close-up video, and voice-over audio based on a received video playback request; The target script data generation unit is configured to insert the voice-over audio into the script data based on the voice-over tags in the script data to generate target script data; The splitting unit is configured to split the target script data based on the off-screen audio and a preset character count threshold to generate segmented target script data; The target segmented video generation unit is configured to determine the state of the virtual avatar used based on the data type of the segmented target script data, and generate a target segmented video based on the shot tags, the state, and the segmented target script data in response to the inclusion of shot tags in the segmented target script data. The video playback unit is configured to generate a playback task based on each of the target video segments, and execute the playback task to complete the playback of the corresponding video.

9. A video playback electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images