Video processing method, device, equipment and medium
Patent Information
- Application Number
- CN202211659371.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-12-22
Smart Images

Figure CN116033221B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, device, equipment and medium. Background Art
[0002] In online classes with three-dimensional virtual scenes, video is a key content. In order to flexibly adapt to the classroom content, it is often necessary to perform secondary editing based on the original recorded video.
[0003] The original video and the inserted new video are synthesized, and then the synthesized video is previewed to determine whether it can be played normally. However, this method cannot flexibly edit the video during the secondary editing process, and the video synthesis
[0004] The synthesis is very time-consuming and resource-consuming. If it cannot be played normally, it needs to be edited and synthesized again, which greatly wastes time and resources. Therefore, there is currently a problem of how to improve the flexibility of video editing and the efficiency of video generation. Summary of the invention
[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a
[0006] A video processing method, device, equipment and medium.
[0007] According to one aspect of the present disclosure, there is provided a video processing method, comprising:
[0008] Obtain the original video and the first video to be replaced;
[0009] When the video preview instruction is obtained, the first video is used as the current video, and the following video 0 processing operation is performed:
[0010] Play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video;
[0011] If they are not equal, the current video is edited according to the comparison result to obtain a new current video;
[0012] Perform the video processing operation at least once until the duration of the new current video is equal to the duration of the clip 5, and determine the new current video as the second video;
[0013] Video synthesis is performed based on the second video and the original video to obtain a target video.
[0014] According to another aspect of the present disclosure, there is provided a video processing device, comprising:
[0015] A video acquisition module, used for acquiring an original video and a first video to be replaced;
[0016] The video preview module is used to, when a video preview instruction is obtained, use the first video as the current video and perform the following video processing operations:
[0017] Play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video;
[0018] If they are not equal, the current video is edited according to the comparison result to obtain a new current video;
[0019] Performing the video processing operation at least once until the duration of the new current video is equal to the clipping time, and determining the new current video as the second video;
[0020] The video synthesis module is used to perform video synthesis based on the second video and the original video to obtain a target video.
[0021] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor;
[0022] a memory for storing instructions executable by the processor;
[0023] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned video processing method.
[0024] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein when the computer instructions are executed on a terminal device, the terminal device implements the above method.
[0025] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0026] The video processing method, device, equipment and medium provided by the embodiments of the present disclosure include: obtaining an original video and a first video to be replaced; when a video preview instruction is obtained, the first video is used as the current video, and the following video processing operations are performed: playing the original video and the current video, and during the playback process, comparing whether the duration of the current video is equal to the preset editing time in the original video; if they are not equal, the current video is edited according to the comparison result to obtain a new current video; performing at least one video processing operation until the duration of the new current video is equal to the editing time, and the new current video is determined as the second video; performing video synthesis based on the second video and the original video to obtain the target video. The present disclosure can adjust the adaptability of the original video and the current video in the time dimension in the video preview stage, reduce the number of video synthesis, and improve the efficiency of video generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0029] Figure 1 A flow chart of a video processing method provided by an embodiment of the present disclosure;
[0030] Figure 2 A video processing operation flow chart provided by an embodiment of the present disclosure;
[0031] Figure 3 Another video processing operation flow chart provided by the embodiment of the present disclosure;
[0032] Figure 4 A schematic diagram of a video processing process provided by an embodiment of the present disclosure;
[0033] Figure 5 A flow chart of a synchronization alignment method provided by an embodiment of the present disclosure;
[0034] Figure 6 A schematic diagram of signaling interaction for synchronization alignment provided in an embodiment of the present disclosure;
[0035] Figure 7 A schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure;
[0036] Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0037] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0038] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0039] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0040] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0041] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0042] In order to improve the flexibility of video editing and the efficiency of video generation, the embodiments of the present disclosure provide a video processing method, apparatus, device and medium. For ease of understanding, the embodiments of the present disclosure are described below.
[0043] Figure 1 The flowchart of a video processing method provided by an embodiment of the present disclosure is shown in FIG. 1 , and the method can be executed by a video processing device, and the device can be implemented by software and / or hardware. Figure 1, the video processing method may include the following steps.
[0044] Step S102, obtaining the original video and the first video to be replaced.
[0045] In this embodiment, the original video may be, for example, a video recorded from a live class. In order to flexibly adapt the original video to the personalized class content, a new first video may be obtained, and the original video may be edited again based on the first video. The first video may be a real-time recorded video, a video uploaded locally, or a video downloaded from the Internet. Figure 4 For the recorded first video and the local first video, in order to avoid the loss of the first video due to operational errors, or in order to reuse the first video, after obtaining the first video, the first video can also be uploaded to the cloud for storage; and then the first video can be downloaded from the cloud during secondary editing.
[0046] Step S104, when a video preview instruction is obtained, the first video is used as the current video, and the following video processing operations are performed.
[0047] In this embodiment, when the user triggers a button, control, icon or other information indicating a video preview, the terminal device can obtain a video preview instruction and start executing at least one video processing operation shown in the following steps S106-S108 according to the video preview instruction.
[0048] Step S106, play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video; if not equal, execute the following step S108; if equal, execute the following step S110.
[0049] In this embodiment, the start time and end time of the secondary editing can be determined in the original video, and the time period from the start time to the end time is the editing time. The original video and the first video are played according to a preset preview mode; wherein the preview mode is: during the editing time, the played video is replaced from the original video to the current video.
[0050] For example, the start time of the original video can be expressed as t0, and the original video's play time ti is recorded in real time according to the original video's play time axis. When the play time reaches the start time (t1) in the clip time, the original video is paused and the current video starts playing; when the current video ends, the original video's play progress is jumped to the end time (t2) in the clip time, and the original video is played from the end time.
[0051] In order to reduce the usage constraints on users, this embodiment has no time limit for the first video, and users can freely collect first videos of any length, or upload first videos of any length locally. However, in the new video formed after the secondary editing of the original video, the length of the first video needs to match the editing time of the original video, so that the videos before and after the switching can be played normally at the beginning and end of the editing time, avoiding video missing due to the first video being too short, or video overlap or video coverage due to the first video being too long. Abnormal playback effects.
[0052] Based on the above considerations, in order to match the duration of the current video with the editing time, this embodiment can compare whether the duration of the current video is equal to the preset editing time in the original video during the process of playing the original video and the current video; if they are not equal, execute the following step S108 to edit the current video; if they are equal, execute the following step S110.
[0053] Step S108, editing the current video according to the comparison result to obtain a new current video.
[0054] In this embodiment, when the duration of the current video is greater than the editing time, a video segment with a duration equal to the editing time is selected from the current video to obtain a new current video.
[0055] When the duration of the current video is less than the editing time, the current video can be deleted, and a new video can be obtained, and a new current video can be determined based on the obtained video. Alternatively, a new video that can fill the missing duration can be obtained based on the current video, and a new current video can be determined based on the current video and the new video.
[0056] For ease of description, the current video can be represented as V i , the new current video can be represented as V i+1 . For the new current video V i+1 , play the original video and the current video V i+1 , and during playback, compare the current video V again i+1 Whether the duration of the new current video is equal to the preset editing time in the original video; ... And so on, the video processing operation is performed multiple times until the duration of the new current video is equal to the editing time, that is, the following step S110 is performed.
[0057] Step S110, when the duration of the new current video is equal to the editing time, the video processing operation is stopped, and the new current video is determined as the second video.
[0058] Specifically, the video processing operation is repeatedly performed until the duration of the new current video is equal to the editing time, and then the new current video obtained when the video processing operation is stopped is determined as the second video.
[0059] In the process of playing the original video and the current video as described above, the original video is not cut, nor is the current video synthesized with the original video. In other words, no editing operation has been performed on the current video and the original video at this time, which is equivalent to playing multiple videos in a replacement manner. Therefore, the video preview process does not require additional consumption of video editing resources. This embodiment does not perform video synthesis in the video preview stage, which reduces the consumption of time and resources, and can facilitate users to adjust the original video, current video, editing time and other editing contents at any time and multiple times until the secondary editing effect corresponding to the preview of the current video and the original video meets the user's requirements.
[0060] Step S112, performing video synthesis based on the second video and the original video to obtain a target video.
[0061] In this embodiment, the video processing operation is performed at least once according to the above method. After the video can be played in replacement according to the editing time, the video durations match each other, and the video content is smoothly connected, the user can initiate a video synthesis instruction. Then, according to the video synthesis instruction, the original video and the second video are synthesized into a target video. The specific synthesis method is, for example, to delete the video segment corresponding to the editing time in the original video, and insert the second video into the editing time in the original video to obtain the target video.
[0062] This embodiment performs video preview before performing video synthesis, which can not only ensure the video synthesis effect, but also reduce the number of video synthesis operations. After the video synthesis effect is met, only one video synthesis operation is required, which significantly reduces the consumption of time and resources.
[0063] The video processing method provided by the embodiment of the present disclosure first obtains the original video and the first video to be replaced. Then, when the video preview instruction is obtained, the first video is used as the current video, and the following video processing operations are performed: the original video and the current video are played, and during the playback process, the duration of the current video is compared to see if it is equal to the preset editing time in the original video; if not, the current video is edited according to the comparison result to obtain a new current video. The above video processing operation is performed at least once until the duration of the new current video is equal to the editing time, and the new current video is determined as the second video. Then, video synthesis is performed based on the second video and the original video to obtain the target video.
[0064] Before video synthesis, this technical solution repeatedly edits the current video by executing at least one video processing operation, so as to facilitate the user to adjust the content of the secondary editing multiple times, until the duration of the current video is equal to the editing time and stops; at this time, it is determined that the second video meets the conditions for video synthesis in the time dimension, and then video synthesis is performed based on the second video and the original video; this method can adjust the adaptability of the original video and the current video in the time dimension during the video preview stage, so that only one video synthesis is required to obtain the target video that meets the playback effect, effectively reducing the number of video synthesis times and the time and resources consumed by unnecessary video synthesis, and improving the video generation efficiency.
[0065] Reference Figure 2 , an embodiment of performing video processing operations is provided herein, including the following contents.
[0066] Step S202, play the original video and the current video V i This embodiment may include: starting to play the original video and obtaining the play time of the original video; when the play time reaches the start time in the clip time, pausing the original video and starting to play the current video; when the current video ends, jumping the play progress of the original video to the end time in the clip time, and playing the original video from the end time.
[0067] Combination Figure 3 Specifically, the original video can be played from t0. When the original video playback time ti=t1, the original video is paused and the current video is played. When the current video is finished, the original video is played again from t2. During the playback process, there is no need to cut the original video or to synthesize the first video with the current video. Multiple videos are played in a replacement manner, so that the video preview process does not need to consume additional video editing resources.
[0068] Considering that the start time and end time in the editing time are the key time points for the connection between the original video and the current video, abnormal playback is most likely to occur. Based on this, in the process of playing the original video and the current video, this embodiment can selectively and flexibly play the video clips corresponding to the above key time points, and several flexible playback embodiments are provided here.
[0069] In one embodiment, the method of starting to play the original video may include: determining a first time before the start time in the editing time according to a preset first time interval on the timeline of the original video; and playing the original video from the first time.
[0070] In actual applications, the original video before the start time t1 generally does not have any playback problems, and only the connection time period between the original video and the current video can be checked, that is, whether it can be played normally before and after the start time t1. Based on this, the first time t1′ before the start time in the editing time is determined according to the first time interval Δt1, that is, t1′=t1-Δt1, and the playback progress of the original video is directly jumped to the first time before the original video is started. When the playback time of the original video reaches the start time t1 in the editing time, the original video is paused and the current video is started; when the current video ends, the playback progress of the original video is jumped to the end time in the editing time, and the original video is played from the end time. The method of this embodiment of playing the original video from the first time before the start time will not miss the key video clips in the original video, and reduces the video preview time.
[0071] Similarly, for the current video, the method for playing the video clips corresponding to its key time points can be: on the timeline of the current video, determine the second time before the end time of the current video according to a preset second time interval; when the playback time of the current video reaches a preset time threshold, jump the playback progress of the current video to the second time; and continue playing the current video from the second time until the playback of the current video ends.
[0072] Specifically, the video clip in the middle of the current video usually does not have any playback problems, and only the connection time period between the current video and the original video can be checked to see whether it can be played normally. In this embodiment, the original video is paused and the current video is started. After the current video is played for a period of time and reaches a preset time threshold (such as 5 seconds), the middle video clip that does not have any playback problems can be skipped, and the playback progress of the current video is jumped to the second time before the end time, and the current video is continued to be played from the second time until the current video ends. At this time, the original video is continued to be played from the end time t2 in the editing time. After the current video starts to play for a period of time, this embodiment directly jumps to the second time before the end time to continue playing. This method will not miss the key video clips in the current video, and reduces the video preview time.
[0073] In another embodiment, during the process of playing the original video and the current video, when a drag instruction is obtained, the playback progress of the original video or the current video is adjusted to the time corresponding to the end position of the drag instruction.
[0074] The present embodiment can drag the video playback progress according to the drag command, and quickly drag the video playback progress to the key time nodes of the video connection such as the start time and end time that need to be paid attention to. With the playback time of the original video as a reference, when the video playback progress is dragged, the playback time of the original video will be updated in real time with the dragging action, and will not affect the replacement playback between the original video and the current video. For example, the original video before the start time t1 generally does not have a playback problem, so the playback progress of the original video is directly dragged to near the start time t1 and then played, and the original video and the current video before and after the start time t1 are checked to play normally. Similarly, for the current video, it can also be dragged to near the end time t2 and then played, and the current video and the original video before and after the end time t2 are checked to play normally. The present embodiment can reduce the video preview time by dragging the video playback progress, and further improve the efficiency of the video secondary editing.
[0075] Step S204, during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video. If they are equal, refer to the following step S210 to stop the video processing operation and determine the second video. If they are not equal, if the duration of the current video is greater than the editing time, execute the following step S206, or if the duration of the current video is less than the editing time, execute the following step S208.
[0076] Step S206, when the duration of the current video is greater than the editing time, a video segment with a duration equal to the editing time is selected from the current video to obtain a new current video.
[0077] In one example, a target time period equal to the editing time can be determined in the current video, and then a video segment corresponding to the target time period can be intercepted in the current video to obtain a new current video with a duration equal to the editing time. For easy distinction, the new current video can be represented as V i+1 .
[0078] Step S208, when the duration of the current video is less than the editing time, delete the current video and obtain a third video.
[0079] In this embodiment, the third video can be a real-time recorded video, a video uploaded from a local computer, or a video downloaded from the Internet, and the third video can be uploaded to the cloud for storage. Considering that the third video is similar to the first video and both are new videos without editing, the processing methods of the two are similar, that is, the third video is also used as the current video and the video processing operation is performed.
[0080] Specifically, for the third video, return to step S202, play the original video and the third video; during the playback process, compare whether the duration of the third video is equal to the editing time; if they are not equal, as in step S206 or S208, edit the third video according to the comparison result to obtain a new current video; if they are equal, execute the following step S210.
[0081] Step S210, stop the video processing operation, and determine the new current video as the second video.
[0082] Repeat steps S202 to S208 until the duration of the current video is equal to the editing time in the original video. At this time, the current video meets the synthesis condition in the time dimension, so it is determined as the second video that can be used for the next step of video synthesis.
[0083] Reference Figure 3 , another embodiment of performing video processing operations is provided herein, including the following contents.
[0084] Step S302, play the original video and the current video V i The specific playing method is referred to the above embodiment and will not be described in detail here.
[0085] Step S304, during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video. If they are equal, execute the following step S314, stop the video processing operation, and determine the second video. If they are not equal, if the duration of the current video is greater than the editing time, execute the following step S306, or, if the duration of the current video is less than the editing time, execute the following steps S308-S312.
[0086] Step S306, when the duration of the current video is greater than the editing time, a video segment with a duration equal to the editing time is selected from the current video to obtain a new current video.
[0087] Step S308, when the duration of the current video is less than the editing time, determine the missing duration between the editing time and the duration of the current video.
[0088] Step S310, obtaining a fourth video whose duration is not less than the missing duration. Specifically, the fourth video whose duration is sufficient to fill the missing duration can be obtained by recording or uploading.
[0089] Step S312: The combination of the current video and the fourth video is used as a new current video.
[0090] This embodiment first determines the missing duration, and then obtains a fourth video that is not less than the missing duration, so that the duration of the combined new current video is not less than the editing time, making it easy to select a video segment equal to the editing time and avoiding too many operations.
[0091] For the new current video V i+1 , return to the above step S302, perform the video processing operation again, repeat this process until the duration of the current video is equal to the editing time, and execute the following step S314.
[0092] Step S314, stop the video processing operation, and determine the new current video as the second video.
[0093] Repeat steps S302 to S314 until the duration of the current video is equal to the editing time in the original video. At this time, the current video meets the synthesis condition in the time dimension, so it is determined as the second video that can be used for the next step of video synthesis.
[0094] like Figure 4 As shown, according to the above embodiment, a second video matching the original video in play time is obtained. In this case, playing the original video and the second video may include: starting to play the original video, when the play time reaches the start time (t1) in the clip time, pausing the original video and starting to play the second video; when the second video finishes playing, jumping the play progress of the original video to the end time (t2) in the clip time, and playing the original video from the end time. Since the duration of the second video is equal to the clip time determined by the start time t1 and the end time t2, when the second video ends playing, the original video should continue to play from the end time t2. In this case, the play progress of the original video is jumped from the time node t1 when it is paused to the time node t2 that needs to be played, that is, the play progress of the original video is jumped to the end time in the clip time, and the original video continues to play from the end time.
[0095] In one embodiment, the second video and the original video may be directly synthesized into a target video.
[0096] In another embodiment, in order to further improve the quality of video synthesis, the following video synthesis method may be provided, including:
[0097] Detect the relevance of video content before and after replacement playback.
[0098] In a specific method, the first video image and the first audio before replacement playback, and the second video image and the second audio after replacement playback are extracted respectively; the similarity of the first video image and the second video image is calculated to obtain a first similarity; the similarity of the first audio and the second audio is calculated to obtain a second similarity; and the correlation between the video contents before and after replacement playback is obtained based on the first similarity and the second similarity.
[0099] When the correlation is higher than the preset correlation threshold, the second video and the original video are synthesized to obtain the target video. When the correlation is higher than the preset correlation threshold, it means that the video content before and after the replacement playback has little change, the content before and after is coherent and harmonious, and there is no abnormal situation such as sudden picture change.
[0100] Based on this, when a video synthesis instruction is obtained, the video segment corresponding to the editing time in the original video is deleted, and the second video is inserted into the editing time in the original video to obtain the target video. In this embodiment, the target video is synthesized again when a video synthesis instruction is obtained, which can reduce the number of video synthesis times and improve the efficiency of video generation. At the same time, it can reserve an operational space for the user, so that the user can operate and process at any time before confirming the video synthesis.
[0101] In an online classroom with a 3D virtual scene, interactive signaling is another key content besides video, such as issuing rewards, calling students, voice interaction, etc., which can increase the fun of the online class. Interactive signaling is used to trigger interactive effects that match the classroom content in the 3D virtual scene as the video plays. Therefore, the triggering time of the interactive signaling needs to be synchronized with the playback time of the video. However, secondary editing of the video may destroy the alignment relationship between the interactive signaling and the original video.
[0102] Based on this, this embodiment needs to synchronize and align the target video with the interactive signaling after the second editing. Currently, there are mainly the following alignment schemes. In scheme one, the video playback progress is calibrated at a frequency of once per second, and whether there is interactive signaling playback at the corresponding time point is calculated based on the video playback time; the video playback in this scheme depends on the network, and there may be freezes and buffering, which causes the problem that the video and interactive signaling cannot be strictly aligned in real time. In scheme two, the video playback can be synchronously controlled through interactive signaling, but in this way, the video playback progress cannot be flexibly dragged.
[0103] In response to the above problems, an embodiment of the present disclosure may provide a method for aligning video and interactive signaling, as described below.
[0104] In this embodiment, the original video is pre-associated with the interactive signaling. Accordingly, the method provided by this embodiment includes: synchronizing the target video with the interactive signaling.
[0105] Among them, the original video has at least one interactive signaling preset, and each interactive signaling carries its own signaling trigger time, that is, the interactive effect corresponding to the interactive signaling will be triggered at the signaling trigger time, for example: issuing rewards at the 10th second, teacher walking at the 20th second, calling students at the 25th second, voice interaction at the 50th second, etc.
[0106] In one method of acquiring interactive signaling, a signaling resource associated with the original video may be acquired; the signaling resource may be parsed to obtain at least one interactive signaling carrying a signaling trigger time. Figure 6 As shown, according to the preset identification number of the original video, a signaling resource can be requested from the backend service, and the signaling resource is, for example, a compressed zip package, which includes at least one interactive signaling. The interactive signaling can be a binary frame signaling stored in a binary format, which can occupy a smaller storage space to record more information. The signaling resource is downloaded and decompressed, and the decompression result is stored in the local sandbox directory. At this point, the preparation work for the interactive signaling is completed.
[0107] Since the target video is obtained by secondary editing based on the original video, there is also a correlation between the target video and the interactive signaling, and the target video and the interactive signaling can be synchronized and aligned.
[0108] Reference Figure 5 In this embodiment, the synchronization alignment of the target video and the interactive signaling can be achieved through the following steps, including:
[0109] Step S502, play the target video, and record the entry time of the target video by starting the local clock service.
[0110] In this embodiment, when the target video starts to be played, the local clock service is started. The local clock service is such as DisplayLink timer. The local clock service can provide standard time information. In this embodiment, the entry time of the target video is recorded by starting the local clock service. It should be noted here that the entry time is the time obtained by recording the time information provided by the local clock service, which indicates the time to enter the target video. In the case of abnormal playback such as freeze, pause, fast forward, etc. of the target video, the entry time is only based on the time information of the local clock service. In contrast, the playback time of the video is based on the playback time axis of the video, and the playback time will change due to abnormal playback such as freeze, pause, fast forward, etc. For example, in the case of a pause in the playback of the target video, the playback time will be paused as the playback is paused, and the entry time will continue to be counted according to the time information provided by the local clock service.
[0111] Step S504: when the entry duration reaches the signaling trigger time carried by the interactive signaling, the current playback time of the target video is calibrated to the signaling trigger time.
[0112] Reference Figure 6 , when the entry duration reaches the signaling trigger time carried by the interactive signaling, the interactive signaling is sent to the pre-configured scene service engine through the local clock service; the scene service engine is, for example, the unity engine. The interactive signaling is parsed by the scene service engine to obtain the signaling trigger time corresponding to the interactive signaling. Specifically, the interactive signaling is parsed by the scene service engine to obtain the information contained in the interactive signaling, such as: the interactive signaling may include the signaling trigger time and the triggered interactive effect, and may also include the trigger position, the triggered virtual object and other information, which is not limited here. The scene service engine parses the signaling trigger time in the interactive signaling, and bridges the signaling trigger time to the video playback channel, so that the video playback channel calibrates the current playback time of the target video to the signaling trigger time.
[0113] In actual applications, the video playback channel is different from the trigger channel of the interactive signaling, so it is necessary to synchronize the target video with the interactive signaling's interactive effects. When the entry duration reaches the signaling trigger time, due to reasons such as freezes, the target video's playback time may not be equal to the entry duration. In this case, in order to synchronize the video playback content with the interactive signaling's interactive effects, the target video's current playback time can be calibrated to the signaling trigger time.
[0114] In this embodiment, the local clock service can send interactive signaling at a higher frequency (15 frames per second), and the scene service engine can parse and send the signaling trigger time with low latency. Therefore, through the mutual cooperation of the local clock service and the scene service engine, the accuracy of the playback time of the target video calibrated based on the signaling trigger time can be improved.
[0115] In a specific embodiment of calibrating the play time, the current play time of the target video can be compared with the signaling trigger time to see if they are the same; if they are not the same, the play progress of the target video is jumped from the current play time to the signaling trigger time. After the play time of the target video is calibrated to the signaling trigger time, the play content of the target video matches the interactive effect of the interactive signaling, and the two are synchronized.
[0116] In this embodiment, when the entry time reaches the signaling trigger time, the interactive effect corresponding to the interactive signaling is triggered. Including: parsing the interactive signaling through the scene service engine to obtain the interactive effect corresponding to the interactive signaling; triggering the interactive effect in the three-dimensional virtual scene through the scene service engine. That is to say, while sending the signaling trigger time to the video playback channel for time calibration through the scene service engine, this embodiment also triggers the interactive effect corresponding to the interactive signaling in the three-dimensional virtual scene through the scene service engine. Take the interactive signaling in the aforementioned embodiment as an example. When the entry time of the target video reaches the 10th second of the signaling trigger time, the current playback time of the target video is calibrated to the 10th second, and the video content at the 10th second is played. At the same time, the interactive effect of issuing rewards is triggered in the three-dimensional virtual scene; when the entry time of the target video reaches the 20th second of the signaling trigger time, the current playback time of the target video is calibrated to the 20th second, and the video content at the 20th second is played. At the same time, the interactive effect of the teacher walking is triggered in the three-dimensional virtual scene.
[0117] In one embodiment, a method for real-time calibration of the current playback time of a target video is provided, as shown below.
[0118] The local clock service generates an empty frame signaling containing the current clock time at a preset frequency (15 frames per second), and sends the empty frame signaling to the scene service engine. Compared with the interactive signaling that includes the signaling trigger time and interactive effects, the empty frame signaling refers to the binary frame signaling that only contains the current clock time. In actual applications, the local clock service sends binary frame signaling to the scene service engine at a frequency of 15 frames per second; it can be understood that at non-signaling trigger time, the binary frame signaling sent by the local clock service is an empty frame signaling, and at the signaling trigger time, the binary frame signaling sent by the local clock service is an interactive signaling.
[0119] The current clock time in the empty frame signaling is parsed by the scene service engine, and the current clock time is bridged to the video playback channel, so that the video playback channel can calibrate the current playback time of the target video to the current clock time. The empty frame signaling does not contain interactive effects, so the empty frame signaling is usually only used to calibrate the current playback time of the target video. During calibration, the current playback time of the target video is obtained; the current playback time is compared with the current clock time to see if they are the same; if they are not the same, the playback progress of the target video is jumped from the current playback time to the current clock time.
[0120] In this embodiment, the sending frequency of empty frame signaling and interactive signaling is relatively high, so that the current playback time of the target video can be calibrated in real time and more accurately. In particular, when the current playback time is calibrated using interactive signaling, the playback time of the video can also be synchronized with the interactive effect at the same time.
[0121] This embodiment synchronizes the target video with the interactive signaling, which can improve the synchronization and consistency between the playback progress of the target video and the interactive effect of the interactive signaling.
[0122] Figure 7 The following is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure. The device can be used to implement the above-mentioned video processing method. The device can be implemented by software and / or hardware. Figure 7 , the video processing device 700 may include:
[0123] The video acquisition module 702 is used to acquire the original video and the first video to be replaced;
[0124] The video preview module 704 is used to, when a video preview instruction is obtained, use the first video as the current video and perform the following video processing operations:
[0125] Play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to the preset editing time in the original video;
[0126] If they are not equal, the current video is edited according to the comparison result to obtain a new current video;
[0127] Performing the video processing operation at least once until the duration of the new current video is equal to the clipping time, and determining the new current video as the second video;
[0128] The video synthesis module 706 is used to perform video synthesis based on the second video and the original video to obtain a target video.
[0129] In some embodiments, the video preview module 704 includes a first clipping unit, which is used to:
[0130] In the case that the duration of the current video is greater than the editing time, a video segment having a duration equal to the editing time is selected from the current video to obtain a new current video.
[0131] In some embodiments, the video preview module 704 includes a second clipping unit, which is used to:
[0132] When the duration of the current video is less than the editing time, deleting the current video and acquiring a third video;
[0133] Comparing whether the duration of the third video is equal to the editing time;
[0134] If they are not equal, the third video is edited according to the comparison result to obtain a new current video.
[0135] In some embodiments, the video preview module 704 includes a third clipping unit, which is used to:
[0136] In the case that the duration of the current video is less than the editing time, determining the missing duration of the difference between the editing time and the duration of the current video;
[0137] Acquire a fourth video whose duration is not less than the missing duration;
[0138] The combination of the current video and the fourth video is used as a new current video.
[0139] In some embodiments, the video preview module 704 includes a video playback unit, which is used to:
[0140] Start playing the original video and obtain the playing time of the original video;
[0141] When the playing time reaches the start time in the clipping time, pausing the playing of the original video and starting to play the current video;
[0142] When the current video ends playing, the playing progress of the original video is jumped to the end time in the clipping time, and the original video is played from the end time.
[0143] In some embodiments, the video playback unit is further configured to:
[0144] On the time axis of the original video, determining a first time before the start time in the clipping time according to a preset first time interval;
[0145] The original video is played starting from the first time.
[0146] In some embodiments, the video playback unit is further configured to:
[0147] On the time axis of the current video, determining a second time before the end time of the current video according to a preset second time interval;
[0148] When the playback time of the current video reaches a preset time threshold, the playback progress of the current video is jumped to the second time;
[0149] Continue playing the current video from the second time until the current video finishes playing.
[0150] In some embodiments, the video playback unit is further configured to:
[0151] During the process of playing the original video and the current video, when a drag instruction is obtained, the playback progress of the original video or the current video is adjusted to the time corresponding to the end position of the drag instruction.
[0152] In some embodiments, the video synthesis module 706 is further configured to:
[0153] Detect the relevance of video content before and after replacement playback;
[0154] When the correlation degree is higher than a preset correlation degree threshold, video synthesis is performed based on the second video and the original video to obtain a target video.
[0155] In some embodiments, the video synthesis module 706 is further configured to:
[0156] When the video synthesis instruction is obtained, the video segment corresponding to the editing time in the original video is deleted, and the second video is inserted into the editing time in the original video to obtain the target video.
[0157] In some embodiments, the original video is pre-associated with interactive signaling; the video processing device 700 further includes an alignment module, which is used to:
[0158] The target video is synchronously aligned with the interactive signaling.
[0159] In some embodiments, the alignment module is further configured to:
[0160] Play the target video, and record the entry time of the target video by starting the local clock service;
[0161] When the entry duration reaches the signaling trigger time carried by the interactive signaling, the current playback time of the target video is calibrated to the signaling trigger time.
[0162] In some embodiments, the alignment module is further configured to:
[0163] Sending the interactive signaling to a pre-configured scene service engine via the local clock service;
[0164] Parsing the interactive signaling by the scene service engine to obtain the signaling triggering time corresponding to the interactive signaling;
[0165] The signaling trigger time is bridged to a video playback channel so that the video playback channel calibrates the current playback time of the target video to the signaling trigger time.
[0166] In some embodiments, the alignment module is further configured to:
[0167] Generate an empty frame signaling including the current clock time at a preset frequency through the local clock service, and send the empty frame signaling to the scene service engine;
[0168] The current clock time in the empty frame signaling is parsed by the scene service engine, and the current clock time is bridged to the video playback channel, so that the video playback channel calibrates the current playback time of the target video to the current clock time.
[0169] In some embodiments, the alignment module is further configured to:
[0170] Compare the current playback time of the target video with the signaling trigger time to see if they are the same;
[0171] If they are not the same, the playback progress of the target video is jumped from the current playback time to the signaling trigger time.
[0172] In some embodiments, the alignment module is further configured to:
[0173] When the entry duration reaches the signaling trigger time, the interactive effect corresponding to the interactive signaling is triggered.
[0174] In some embodiments, the alignment module is further configured to:
[0175] Parsing the interactive signaling through a scene service engine to obtain an interactive effect corresponding to the interactive signaling;
[0176] The interactive effect is triggered in the three-dimensional virtual scene through the scene service engine.
[0177] In some embodiments, the alignment module is further configured to:
[0178] Acquire a signaling resource associated with the original video;
[0179] The signaling resource is parsed to obtain at least one interactive signaling carrying a signaling trigger time.
[0180] The implementation principle and technical effects of the device provided in this embodiment are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0181] The exemplary embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to cause the electronic device to perform the method according to the embodiment of the present disclosure when executed by the at least one processor.
[0182] The exemplary embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform the method according to the embodiments of the present disclosure.
[0183] refer to Figure 8 , a block diagram of an electronic device 800 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0184] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0185] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 may be any type of device capable of inputting information to the electronic device 800, and the input unit 806 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 807 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 808 may include, but is not limited to, a disk, an optical disk. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0186] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above. For example, in some embodiments, the video processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. In some embodiments, the computing unit 801 may be configured to perform the video processing method in any other appropriate manner (e.g., by means of firmware).
[0187] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0188] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0189] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0190] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0191] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0192] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0193] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A video processing method, characterized in that: include: Acquire an original video and a first video to be replaced; wherein the first video is a new video independent of the original video, and the first video is a video of any length; When the video preview instruction is obtained, the first video is used as the current video, and the following video processing operations are performed: Play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to a preset editing time in the original video; the editing time is a predetermined time in the original video that needs to be edited twice; If they are not equal, the current video is edited according to the comparison result to obtain a new current video; Performing the video processing operation at least once until the duration of the new current video is equal to the clipping time, and determining the new current video as the second video; Video synthesis is performed based on the second video and the original video to obtain a target video.
2. The method according to claim 1, characterized in that The step of editing the current video according to the comparison result to obtain a new current video includes: In the case that the duration of the current video is greater than the editing time, a video segment having a duration equal to the editing time is selected from the current video to obtain a new current video.
3. The method according to claim 1, characterized in that The step of editing the current video according to the comparison result to obtain a new current video includes: When the duration of the current video is less than the editing time, deleting the current video and acquiring a third video; Comparing whether the duration of the third video is equal to the editing time; If they are not equal, the third video is edited according to the comparison result to obtain a new current video.
4. The method according to claim 1, characterized in that: The step of editing the current video according to the comparison result to obtain a new current video includes: In the case that the duration of the current video is less than the editing time, determining the missing duration of the difference between the editing time and the duration of the current video; Acquire a fourth video whose duration is not less than the missing duration; The combination of the current video and the fourth video is used as a new current video.
5. The method according to claim 1, characterized in that The playing of the original video and the current video includes: Start playing the original video and obtain the playing time of the original video; When the playing time reaches the start time in the clipping time, pausing the playing of the original video and starting to play the current video; When the current video ends playing, the playing progress of the original video is jumped to the end time in the clipping time, and the original video is played from the end time.
6. The method according to claim 5, characterized in that The starting to play the original video includes: On the time axis of the original video, determining a first time before the start time in the clipping time according to a preset first time interval; The original video is played starting from the first time.
7. The method according to claim 5, characterized in that The method further comprises: On the time axis of the current video, determining a second time before the end time of the current video according to a preset second time interval; When the playback time of the current video reaches a preset time threshold, the playback progress of the current video is jumped to the second time; Continue playing the current video from the second time until the current video finishes playing.
8. The method according to claim 1, characterized in that The method further comprises: During the process of playing the original video and the current video, when a drag instruction is obtained, the playback progress of the original video or the current video is adjusted to the time corresponding to the end position of the drag instruction.
9. The method according to claim 1, characterized in that: The step of performing video synthesis based on the second video and the original video to obtain a target video includes: Detect the relevance of video content before and after replacement playback; When the correlation degree is higher than a preset correlation degree threshold, video synthesis is performed based on the second video and the original video to obtain a target video.
10. The method according to claim 1 or 9, characterized in that: The step of performing video synthesis based on the second video and the original video to obtain a target video includes: When the video synthesis instruction is obtained, the video segment corresponding to the editing time in the original video is deleted, and the second video is inserted into the editing time in the original video to obtain the target video.
11. The method according to claim 1, characterized in that: The original video is pre-associated with interactive signaling; the method further includes: The target video is synchronously aligned with the interactive signaling.
12. The method according to claim 11, characterized in that The synchronously aligning the target video with the interactive signaling includes: Play the target video, and record the entry time of the target video by starting the local clock service; When the entry duration reaches the signaling trigger time carried by the interactive signaling, the current playback time of the target video is calibrated to the signaling trigger time.
13. The method according to claim 12, characterized in that The step of calibrating the current playback time of the target video to the signaling trigger time includes: Sending the interactive signaling to a pre-configured scene service engine via the local clock service; Parsing the interactive signaling by the scene service engine to obtain the signaling triggering time corresponding to the interactive signaling; The signaling trigger time is bridged to a video playback channel so that the video playback channel calibrates the current playback time of the target video to the signaling trigger time.
14. The method according to claim 13, characterized in that The method further comprises: Generate an empty frame signaling including the current clock time at a preset frequency through the local clock service, and send the empty frame signaling to the scene service engine; The current clock time in the empty frame signaling is parsed by the scene service engine, and the current clock time is bridged to the video playback channel so that the video playback channel calibrates the current playback time of the target video to the current clock time.
15. The method according to any one of claims 13-14, characterized in that: The step of calibrating the current playback time of the target video to the signaling trigger time includes: Compare the current playback time of the target video with the signaling trigger time to see if they are the same; If they are not the same, the playback progress of the target video is jumped from the current playback time to the signaling trigger time.
16. The method according to claim 12, characterized in that The method further comprises: When the entry duration reaches the signaling trigger time, the interactive effect corresponding to the interactive signaling is triggered.
17. The method according to claim 14, characterized in that The interactive effect corresponding to the triggering of the interactive signaling includes: Parsing the interactive signaling through a scene service engine to obtain an interactive effect corresponding to the interactive signaling; The interactive effect is triggered in the three-dimensional virtual scene through the scene service engine.
18. The method according to claim 11, characterized in that The method further comprises: Acquire a signaling resource associated with the original video; The signaling resource is parsed to obtain at least one interactive signaling carrying a signaling trigger time.
19. A video processing device, characterized in that: include: A video acquisition module, used to acquire an original video and a first video to be replaced; wherein the first video is a new video independent of the original video, and the first video is a video of any length; The video preview module is used to, when a video preview instruction is obtained, use the first video as the current video and perform the following video processing operations: Play the original video and the current video, and during the playback process, compare whether the duration of the current video is equal to a preset editing time in the original video; the editing time is a predetermined time in the original video that needs to be edited twice; If they are not equal, the current video is edited according to the comparison result to obtain a new current video; Performing the video processing operation at least once until the duration of the new current video is equal to the clipping time, and determining the new current video as the second video; The video synthesis module is used to perform video synthesis based on the second video and the original video to obtain a target video.
20. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the video processing method described in any one of claims 1-18.
21. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 18.
