Video processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a video processing method and device and electronic equipment, and the method comprises the steps: segmenting a time axis for video editing into a plurality of time periods according to a material track segment on an editing track in a video editing process; performing synthesis rendering on a material track fragment located in a target time period in the plurality of time periods on the editing track, generating video rendering data of the target time period, and caching the video rendering data; and in response to a received video editing preview instruction, if a preview position is located in the target time period, reading rendering data at the preview position from the cached video rendering data in the target time period and displaying the rendering data. According to the embodiment of the invention, the jamming phenomenon caused by real-time rendering when the editing effect is previewed in the video editing process can be reduced, and thus the user experience in the video editing process can be improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a video processing method, device, and electronic device. Background Art

[0002] Video editing usually includes the process of adding special effects, scenes to recorded videos, or mixing video clips from different video sources to obtain new videos.

[0003] The above-mentioned video editing and editing effect preview process requires a large amount of system resources. When the electronic device performing video editing displays the editing effect, the screen usually freezes. In the process of video editing, users usually preview the editing effect frequently, and will encounter the problem of screen freeze when previewing the editing effect multiple times. Therefore, the user's video editing experience is poor. Summary of the invention

[0004] The embodiments of the present disclosure provide a video processing method, device and electronic device to overcome the problem of poor experience caused by the freeze of preview editing effect during video editing in the related art.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising: during a video editing process, dividing a timeline used for video editing into multiple time periods according to material track segments on an editing track; performing synthetic rendering on the material track segments on the editing track that are within a target time period among the multiple time periods, generating and caching video rendering data for the target time segment; in response to receiving a video editing preview instruction, if a preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period.

[0006] In a second aspect, an embodiment of the present disclosure provides an image processing device, comprising: a segmentation unit, for segmenting a timeline used for video editing into multiple time periods according to material track segments on an editing track during a video editing process; a rendering unit, for synthesizing and rendering the material track segments on the editing track that are located in a target time period among the multiple time periods, generating and caching video rendering data for the target time segment; and a display unit, for responding to receiving a video editing preview instruction, and if a preview position is located in the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the first aspect above and various video processing methods that may be involved in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and various video processing methods that may be involved in the first aspect.

[0010] The video processing method, device and electronic device provided by the present embodiment divide the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process; perform synthetic rendering on the material track segments on the editing track that are located in a target time period among the multiple time periods, generate and cache video rendering data of the target time segment; in response to receiving a video editing preview instruction, if the preview position is located in the target time period, read and display the rendering data of the preview position from the video rendering data cached in the target time period. Since the timeline is divided into multiple time periods according to the material track segments on the editing track, and then the material track segments in the target time period are synthetically rendered, when previewing the video editing effect, a preview video is generated according to the video rendering data corresponding to different target time periods, which can reduce the jamming phenomenon caused by the need for real-time rendering when previewing the editing effect during the video editing process, thereby improving the user experience during the video editing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 A schematic diagram of a video editing interface provided by an embodiment of the present disclosure;

[0013] Figure 2 A video processing method according to an embodiment of the present invention is provided. Figure 1 ;

[0014] Figure 3 A schematic diagram of timeline segmentation provided in an embodiment of the present disclosure;

[0015] Figure 4 A video processing method according to an embodiment of the present invention is provided. Figure 2 ;

[0016] Figure 5 A schematic diagram of a scenario provided for the present disclosure;

[0017] Figure 6 A schematic diagram of a scenario provided for the present disclosure;

[0018] Figure 7 A structural block diagram of a video processing device provided by an embodiment of the present disclosure;

[0019] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0021] Please refer to Figure 1 , which shows a schematic diagram of a video editing. The schematic diagram shows a process of mixing two video materials. The video editing mixes the video material 10 with the video material 20. The timeline 13 of the video editing interface may include tracks 11 and 12 below. Track 11 includes a material track segment consisting of multiple frames selected from the video material 10. Track 12 may display a material track segment consisting of multiple frames selected from the video material 20.

[0022] The user drags the material track segment at track 11 and the material track segment at video frame area 12 to select the frame at which the video material 10 and the video material 20 start to be mixed. After determining multiple mixed frames of the two, the effect can be previewed to browse the editing effect.

[0023] Figure 1This is just an example of video editing. In practice, you can mix multiple video materials, or add text, special effects graphics, etc. to a video. During the video editing process, users usually preview the editing effects in order to view the editing effects. If the electronic device used by the user for video editing has a low resource configuration, or the material used for video editing is large, when previewing the editing effect, each frame needs to be rendered and a preview screen is generated based on the rendering result. Since the rendering of each frame corresponding to the editing result itself requires more system resources, the preview editing result will be displayed in a stuck state, resulting in a poor user experience.

[0024] In order to improve the above-mentioned problem, the solution provided by the present invention divides the timeline into multiple time periods according to the material track segments on the editing track, generates and caches video rendering data for a target time period among the multiple time periods, and the video rendering data of the target time period can be used to generate a preview screen during preview. In addition, the video rendering data processed in the above manner can be reused, which can reduce the resources required for previewing the video editing effects, thereby reducing the jamming phenomenon.

[0025] Please refer to Figure 2 , Figure 2 A video processing method according to an embodiment of the present invention is provided. Figure 1 The video processing method in the present disclosure may be executed by a terminal device, and the method includes:

[0026] S201: During the video editing process, a timeline for video editing is divided into a plurality of time periods according to material track segments on an editing track.

[0027] In this embodiment, various video editing applications can be run in the terminal device, in which video materials to be edited can be imported, and then editing operations can be performed on the video materials.

[0028] The video material to be edited may include video material obtained from various methods, including but not limited to: video material recorded using a video recording device, video material obtained from other electronic devices based on authorization, etc. The video material may also include one of the following: preset special effects, preset background, etc. The video material to be edited is described here only by way of example, and not all video materials are exhaustive. The above examples are not intended to limit the protection scope of the disclosed solution.

[0029] Usually in video editing applications, you can display a timeline in the video editing interface. The timeline generally refers to the horizontal list of thumbnails that appear below the video preview when editing a video and move as the video plays. The timeline can be used to control the screen content displayed at a certain time point in the video. In addition, the timeline and network technology can be used to connect multiple events in chronological order to form a relatively complete recording system.

[0030] For a video clip in the video editing material, after the video clip is transcoded, multiple video frames corresponding to the video clip are displayed below the timeline in an order of appearance time. Figure 1 As shown, for video 1, multiple video frames of the video material 10 can be displayed below the timeline 13.

[0031] During video editing, an editing track may include at least one material track clip.

[0032] The material track clips include text material track clips, graphic special effect material track clips or video material track clips.

[0033] The above timeline can be split according to the material track segments on the editing track to obtain multiple time periods.

[0034] As an implementation method, the start point and end point of the material track segment can be used as the split point of the time axis, and the time axis can be split according to the split points. For multiple material track segments, the start point and end point of each material track segment can be used as the split point to split the time axis. For material track segments in different tracks, the start point and end point of each material track segment can be used as the split point of the time axis.

[0035] By dividing the time axis through the above-mentioned division points, multiple time periods can be obtained.

[0036] In this way, the starting point of each material track segment is the starting point of a time period and the end point of the previous time period of this time period;

[0037] In the above time segmentation method, the end point of each material track segment is the end point of a time segment and the starting point of the next time segment after the time segment.

[0038] In each time segment, there will not be more than one material track segment data in the same track in the time segment.

[0039] Please refer to Figure 3 , Figure 3A schematic diagram of the division of a timeline is shown. Different video materials can correspond to different tracks, for example, video material 1 and video material 2 can correspond to track 1 and track 2 respectively. Assume that the first frame of track material segment 1 appears on track 1 at 00:00 (t0) on the timeline (that is, t0 is the starting point of track material segment 1), and the first frame of the corresponding material track segment 2 in video material 2 appears at t1 (t1 is the starting point of track material segment 2), and at time t2, material track segment 1 ends (t2 is the ending point of material track segment 1); at time t3, material track segment 2 ends (t3 is the ending point of material track segment 2). In this way, since the timeline can be divided with t0, t1, t2, and t3 as the division points, the time period 1 corresponding to t0-t1; the time period 2 corresponding to t1-t2; and the time period 3 corresponding to t2-t3 are obtained. t1 is the ending point of time period 1 and also the starting point of time period 2. t2 is the starting point of time period 3 and also the ending point of time period 2.

[0040] The multiple time periods obtained by the above segmentation method have good continuity of the material track segments in each track in the time period. In addition, each time period obtained by segmentation in the above method will not include multiple material track segments, so the amount of calculation required to generate video rendering data for each time period is small, which helps to quickly generate video rendering data for each time period.

[0041] S202: Perform synthesis rendering on the material track segments in the target time segment among the multiple time segments on the editing track, generate video rendering data of the target time segment, and cache it.

[0042] After multiple time periods are obtained in step S101, a target time period among the multiple time periods may be determined. The target time period may be a time period corresponding to a larger number of material track segments among the multiple time periods, or a time period corresponding to a material track segment with more picture information changes.

[0043] For the target time period, the material track segments corresponding to the target time period can be determined, and then a synthesis rendering task can be generated. The synthesis rendering task can then be executed. Specifically, the material track segments corresponding to the target time period can be synthesized according to a preset synthesis rule, and then rendered to obtain video rendering data corresponding to the target time period.

[0044] For example, the video frames corresponding to the material track segments aligned on the time axis in the target time period are synthesized in units of frames and then rendered. Alternatively, the video frames and graphic frames in the material track segments aligned on the time axis are synthesized in units of frames and then rendered. Therefore, the video rendering data corresponding to the target time period may include synthesized rendering data corresponding to multiple frames belonging to the target time period.

[0045] In practice, the rendering control management object provided by the video editing application can be called to perform the composite rendering task.

[0046] The composite rendering tasks corresponding to the multiple target time periods may be executed sequentially in a serial task execution manner or in parallel in a parallel task execution manner.

[0047] For each target time period, the video rendering data corresponding to the target time period may be cached.

[0048] In some embodiments, the above-mentioned video processing method further includes: caching the video rendering data using a global storage path.

[0049] In these embodiments, the global storage path here may be a storage path other than the draft corresponding to the video material being edited.

[0050] After setting the global storage path, you can associate the video rendering data with the draft of the above video material. In this way, when the draft name is modified, the video rendering data will not be affected by the change of the draft name.

[0051] S203: In response to receiving the video editing preview instruction, if the preview position is within the target time period, read and display the rendering data of the preview position from the video rendering data cached in the target time period.

[0052] When editing a video, the user often needs to browse the editing effect. During the editing operation, the user can click on the preview control that can be used to preview the editing effect, thereby sending the preview instruction to the execution subject.

[0053] After receiving the preview instruction, the preview position indicated by the preview instruction can be determined. If the preview position indicated by the preview instruction is within the target time period, the rendering data corresponding to the preview position is read from the video rendering data cached in the target time period and displayed.

[0054] In this embodiment, during the video editing process, the timeline used for video editing is divided into multiple time periods according to the material track segments on the editing track; the material track segments on the editing track that are located in the target time period among the multiple time periods are synthesized and rendered, and the video rendering data of the target time segment is generated and cached; in response to receiving a video editing preview instruction, if the preview position is located in the target time period, the rendering data of the preview position is read from the video rendering data cached in the target time period and displayed. Since the timeline is divided into multiple time periods according to the material track segments on the editing track, and then the material track segments in the target time period among the multiple time periods are synthesized and rendered, when previewing the video editing effect, a preview screen is generated according to the video rendering data corresponding to the target time period, which can reduce the jamming phenomenon caused by the need for real-time rendering when previewing the editing effect during the video editing process, and thus improve the user experience during the video editing process.

[0055] Please refer to Figure 4 , FIG. 1 is a flow chart of a video processing method provided by an embodiment of the present disclosure Figure 2 The video processing method in the present disclosure may be executed by a terminal device, and the method includes:

[0056] S401: During the video editing process, a timeline for video editing is divided into a plurality of time periods according to material track segments on an editing track.

[0057] S402: Perform synthesis rendering on the material track segments in the target time segment among the multiple time segments on the editing track, generate video rendering data of the target time segment, and cache it.

[0058] The specific implementation of steps S401 to S402 can refer to Figure 2 The relevant description of the illustrated embodiment is not repeated here.

[0059] S403: For each target time period, generate a first identifier corresponding to the target time period according to the picture data of the material track segment corresponding to the target time period; and set a first identifier for the cached video rendering data corresponding to the target time period.

[0060] The target time period may be a time period among the multiple time periods obtained in step S401. Schematically, the target time period may be a time period determined from the multiple time periods according to the user's selection operation. It may also be a time period determined according to the amount of screen information change in each time period.

[0061] The first identifier here may be a first identifier obtained using various digital signature algorithms.

[0062] Indicatively, for Figure 3 For time period 2 (target time period), a digital signature algorithm can be used for the picture data in material track segment 1 and material track segment 2 corresponding to time period 2, thereby obtaining a first identifier for time period 2. The first identifier obtained by the above digital signature algorithm is usually a unique identifier.

[0063] Illustratively, the above digital signature algorithm may be a message digest algorithm (Message Digest Algorithm MD5).

[0064] The first identifier is unique and irreversible. When the video editing data input to the digital signature algorithm changes, such as adding or reducing video frames, or adding special effects or backgrounds, the first identifier obtained by the digital signature algorithm will change.

[0065] Through the above steps, for a target time period among multiple time periods, the track material segments within the target time period can be synthesized and rendered to obtain video rendering data. The video rendering data of the target time period is cached and the above first identifier is set. Since the first identifier is unique, the above video rendering data can be accurately found in the cache by the above first identifier corresponding to the target time period.

[0066] For example, for Figure 2 In the time period 2 in the buffer, each frame of picture data of the material track segment 1 of the corresponding video material 1 in the time period 2 and each frame of picture data of the material track segment 2 of the corresponding video material 2 can be synthesized and rendered in advance to obtain the video rendering data of the time period 2. The first identifier of the time period 2 can be determined by using a digital signature algorithm for the multiple frames of user data in the material track segment 1 in the time period 2 and the multiple frames of picture data in the material track segment 2, and the first identifier is set for the cached video rendering data of the time period 2.

[0067] Therefore, for the same time period, as long as the picture data of the material track segments of each track in the time period is not changed, the first identifier calculated using the same digital signature algorithm for the picture data in the material track segments in the time period is the same. Based on this, when previewing the editing effect, the video rendering data corresponding to each target time period can be accurately found according to the first identifier.

[0068] In addition, adding the first identifier to the identifier of the video rendering data of the target time period can avoid performing multiple synthesis rendering on the same material track segment in the same time period.

[0069] S404: In response to receiving the video editing preview instruction, if the preview position is within the target time period, determine a first identifier corresponding to the target time period.

[0070] S405: Determine rendering data corresponding to a preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0071] After receiving the video editing preview instruction, the time period in which the video editing preview instruction falls can be determined. For example, the first identifier of the time period can be generated according to the screen data of each material editing segment in the time period according to the algorithm for generating the first identifier, and then the first identifier is used to match in the cache. If the match is successful, it means that the time period is the target time period. Then, the video rendering data corresponding to the target time period is extracted from the cache.

[0072] Then, the rendering data corresponding to the preview position is extracted from the video rendering data corresponding to the target time period.

[0073] In this embodiment, compared with Figure 2 The solution of the embodiment shown generates a first identifier related to the material track segment screen data for the target time period and caches the video rendering data of the target time period according to the first identifier. Therefore, when receiving a preview instruction, the video rendering data of the target time period can be accurately and quickly found from the cache according to the first identifier, and the rendering data corresponding to the preview position can be extracted. The accuracy of the preview effect displayed by the preview instruction can be improved.

[0074] In some embodiments, the video processing method further includes: for the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and the above step S405 further includes the following steps:

[0075] First, according to the first identifier of the target time period, the video rendering data corresponding to the target time period is determined from the cache.

[0076] Secondly, the rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the above mapping relationship.

[0077] Here, by establishing a mapping relationship between video rendering data and the frame information of the material track segment, it is possible to clarify which frames in the material a video rendering data corresponds to. When the preview instruction indicates the rendering data corresponding to a target time period, the video rendering data corresponding to the target time period can be first located according to the first identifier, and then the rendering data corresponding to the preview position indicated by the preview instruction can be extracted from the video rendering data according to the frame information, so that the editing effect corresponding to the preview position can be accurately and quickly displayed.

[0078] In some embodiments, the above step S404 includes the following steps:

[0079] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0080] A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated from the frame information.

[0081] Indicatively, for Figure 3 The video rendering data in time period 1 in the target time period, the frame information of time period 1 can be the frame information "1-5 frames of video images" of the material track segment in the video material 1. The above second identifier can be, for example, "video material 1 video rendering data corresponding frame (1-5.mp4)". By generating a second identifier including the above frame information for the target time period, and setting the above second identifier as a part of the identifier of the video rendering data of the target time period. Thus, a mapping relationship between the video rendering data of the target time period and the frame information corresponding to the target time period is established.

[0082] In these embodiments, a second identifier generated by frame information can be set for the video rendering data. Then, the rendering data corresponding to the video frame in the target time period can be quickly determined according to the second identifier, so the rendering data corresponding to the preview position can be quickly determined from the cache according to the second identifier, which can further improve the speed of previewing the editing effect.

[0083] In some embodiments, the above-mentioned determining the frame information of the material track segment corresponding to the target time period in the corresponding material includes:

[0084] For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0085] In these embodiments, the editing track may include multiple tracks, and the material may include multiple materials.

[0086] Combine the following Figure 5 To explain, Figure 5A schematic diagram of an application scenario provided by the present disclosure. During the editing process, the material track segment 3 corresponding to the target time segment D on the timeline is a segment from the 100th frame to the 200th frame extracted from the video material 1. The material track segment 4 is a material track segment extracted from the video material 2. Among them, the material track segment of the video material 2 corresponding to the target time segment D is a segment from the 75th frame to the 175th frame of the video material 2. The user performs a mixed editing operation on the material track segment 3 and the material track segment 4. On the editing track, the appearance time of the material track segment 4 is later than the appearance time of the material track segment 3. The first frame of the video material 4 can be used as the starting position for determining the frame information of the target time segment D. This is because the video material 1 and the video material 2 are mixed at most from the first frame of the video material 2.

[0087] In addition, a first identifier E can be generated based on the picture data of the 100th to 200th frames of video material 1 and the 75th to 175th frames of video material 2. A second identifier can be generated based on the frame information of the 75th to 175th frames of video material 2. Video rendering data can be generated for the material track segment 3 and the material track segment 4 corresponding to the above-mentioned target time period D, and a first identifier and a second identifier are set for the video rendering data. A mapping relationship between the frame information 75-175 and the video rendering data is established using the second identifier. The second identifier can be, for example, "75-175.mp4".

[0088] In these embodiments, in the scenario of editing multi-track video materials, for a target time period, the frame information of the track material segment that appears the latest on the time axis in the corresponding material is used to determine the frame mapping information of the video rendering data and the track material segment of the target time period, so that when the user performs subsequent editing operations on the same video material, the video rendering data generated by the historical editing operation can be reused, thereby saving system resources.

[0089] In some embodiments, the video processing method further includes the following steps:

[0090] First, in response to receiving a re-editing operation performed on a material track segment, the timeline is segmented according to information of the material track segment on the editing track.

[0091] Secondly, a new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0092] In these embodiments, video editing usually requires multiple editing operations on the video material to finally generate a video editing result that satisfies the user.

[0093] After the video rendering data is generated according to the previous editing operation and previewed, the user can continue to perform editing operations on the video editing material.

[0094] After receiving the user's re-editing operation on the video editing material, the time axis is segmented again according to the segmentation process of step S401, and multiple segmentation results can be obtained.

[0095] For each segmentation result, it can be determined whether the segmentation result overlaps with the historical target time period. If so, a new target time period and corresponding video rendering data are not generated. If part of the segmentation result overlaps with the historical target time period, a new target time period can be generated based on the non-overlapping part. If there is no part of the segmentation result that overlaps with the historical target time period, the segmentation result can be used as the new target time period.

[0096] For the new target time period, video rendering data is generated and cached. When caching, a first identifier and a second identifier are set for the video rendering data. The first identifier is generated by the picture data of the new target time period, and the second identifier is generated by the frame information of the material track segment corresponding to the new target time period in the corresponding material.

[0097] When a preview instruction for previewing the re-editing operation is received, for a target time period that coincides with the historical target time period, the video rendering data corresponding to the historical target time period may be reused for preview presentation.

[0098] Combination Figure 5 and Figure 6 , Figure 6 A schematic diagram of an application scenario provided by the present disclosure. Figure 5 In the example, the user performs a mixed editing operation on the 100th to 200th frames of video material 1 and the 75th to 175th frames of video material 2. In the subsequent editing operation, video material 1 is moved to the left on the time axis, and the number of frames for mixing video material 1 and video material 2 is increased. Figure 6 As shown in FIG. 1 , after the user's re-editing operation, the 75th to 200th frames of the video material 1 are mixed with the 50th to 175th frames of the video material 2. That is, the video material 1 corresponds to the material track segment 3', and the video material 2 corresponds to the material track segment 4. Figure 6 shown.

[0099] for Figure 6The editing operation shown can generate a new target time period F corresponding to the 75th to 99th frames of the video material 1 in the above manner. For the new target time period F, a first identifier P is generated based on the picture data of the 75th to 99th frames of the video material 1 and the 50th to 74th frames of the video material 2. A second identifier of the new target time period is generated based on the frame information of the track material segment 4 corresponding to the new target time period F in the video material 2. Video rendering data of the new target time period is generated, and a mapping relationship between the video rendering data and the frame information corresponding to the new target time period is established using the second identifier. The above-mentioned video rendering data information is cached, and identifiers including the first identifier and the second identifier are used for the video rendering data.

[0100] For the 100th to 200th frames of the video material 1 and the 75th to 175th frames of the video material 2, the video rendering data corresponding to the original target time period D can be reused. Figure 6 When you preview the editing operation of Figure 5 The video rendering data corresponding to the target time period D generated by the historical editing operation shown is previewed. When the preview position is located at the target time period F, the new video rendering data generated this time can also be used for preview.

[0101] In these embodiments, for multiple editing operations, the video rendering data corresponding to the original historical target time period can be reused, thereby reducing the resources required for rendering the multiple editing operations separately, and allowing the user to display the preview screen generated according to the video rendering data more smoothly each time the user previews.

[0102] Corresponding to the video processing method of the above embodiment, Figure 7 The structure block diagram of the video processing device provided by the embodiment of the present disclosure. For the convenience of explanation, only the part related to the embodiment of the present disclosure is shown. Figure 7 The device 70 includes: a segmentation unit 701, a rendering unit 702 and a display unit 703. Among them:

[0103] The segmentation unit 701 is used to segment the time axis used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process;

[0104] A rendering unit 702 is used to perform synthesis rendering on the material track segments in a target time segment among multiple time segments on the editing track, generate video rendering data of the target time segment and cache it;

[0105] The display unit 703 is used for reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period in response to receiving the video editing preview instruction if the preview position is within the target time period.

[0106] In some embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period and is also the end point of the previous time period of the time period;

[0107] The end point of each material track segment is the end point of a time period and the starting point of the next time period after the time period.

[0108] In some embodiments of the present disclosure, the apparatus 70 further includes an identification configuration unit (not shown in the figure). The identification configuration unit is used to:

[0109] For each target time period, a first identifier corresponding to the target time period is generated according to the picture data of the material track segment corresponding to the target time period; and a first identifier is set for the cached video rendering data corresponding to the target time period.

[0110] In some embodiments of the present disclosure, the display unit 703 is further configured to:

[0111] Determine a first identifier corresponding to the target time period;

[0112] The rendering data corresponding to the preset preview position is determined from the video rendering data corresponding to the first identifier read from the cache.

[0113] In some embodiments of the present disclosure, the apparatus 70 further includes a mapping unit (not shown in the figure), which is configured to:

[0114] For the video rendering data of the target time period, a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period is established; and the display unit 703 is further used to:

[0115] Determining, from a cache, video rendering data corresponding to the target time period according to a first identifier of the target time period;

[0116] The rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the mapping relationship.

[0117] In some embodiments of the present disclosure, the mapping unit is further configured to:

[0118] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0119] A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated from the frame information.

[0120] In some embodiments of the present disclosure, the mapping unit is further configured to:

[0121] For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0122] In some embodiments of the present disclosure, the segmentation unit is further configured to:

[0123] In response to receiving a re-editing operation performed on the material track segment, splitting the timeline according to information of the material track segment on the editing track;

[0124] Rendering units are further used to:

[0125] A new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0126] In some embodiments of the present disclosure, the device 70 further includes a storage unit (not shown in the figure), and the storage unit is used to:

[0127] Use the global storage path to store video rendering data.

[0128] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0129] refer to Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing an embodiment of the present disclosure, and the electronic device 800 may be various terminal devices and servers. Among them, the terminal device may include an extended reality device. The extended reality device may include, for example, various wearable devices, such as a head-mounted extended reality device, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0130] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0131] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0132] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0133] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0134] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0135] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0136] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of the unit does not limit the unit itself in some cases. For example, the segmentation unit may also be described as "a unit that segments the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process."

[0139] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0140] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] In a first aspect, according to one or more embodiments of the present disclosure, a video processing method is provided, including:

[0142] During the video editing process, the timeline used for video editing is divided into multiple time periods according to the material track clips on the editing track;

[0143] Performing composite rendering on the material track segments in the target time segment among the multiple time segments on the editing track, generating and caching video rendering data of the target time segment;

[0144] In response to receiving the video editing preview instruction, if the preview position is within the target time period, the rendering data of the preview position is read from the video rendering data buffered in the target time period and displayed.

[0145] According to one or more embodiments of the present disclosure,

[0146] The starting point of each material track segment is the starting point of a time period, and is also the end point of the previous time period of this time period;

[0147] The end point of each material track segment is the end point of a time period and the starting point of the next time period after the time period.

[0148] According to one or more embodiments of the present disclosure, the method further includes: for each target time period, generating a first identifier corresponding to the target time period based on the picture data of the material track segment corresponding to the target time period; and setting a first identifier for the cached video rendering data corresponding to the target time period.

[0149] According to one or more embodiments of the present disclosure, in response to receiving a video editing preview instruction, if the preview position is within a target time period, reading and displaying rendering data of the preview position from the video rendering data cached in the target time period includes:

[0150] Determine a first identifier corresponding to the target time period;

[0151] The rendering data corresponding to the preset preview position is determined from the video rendering data corresponding to the first identifier read from the cache.

[0152] According to one or more embodiments of the present disclosure, the method further includes:

[0153] For the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and

[0154] Determining rendering data corresponding to the preview position from the video rendering data corresponding to the first identifier read from the cache includes:

[0155] Determining, from a cache, video rendering data corresponding to the target time period according to a first identifier of the target time period;

[0156] The rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the mapping relationship.

[0157] According to one or more embodiments of the present disclosure, for rendering data of each time period, establishing a mapping relationship between the rendering data and frame information of a material track segment in the time period includes:

[0158] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0159] A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated from the frame information.

[0160] According to one or more embodiments of the present disclosure, determining frame information of a material track segment corresponding to the target time period in a corresponding material includes:

[0161] For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0162] According to one or more embodiments of the present disclosure, the method further includes:

[0163] In response to receiving a re-editing operation performed on the material track segment, splitting the timeline according to information of the material track segment on the editing track;

[0164] A new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0165] According to one or more embodiments of the present disclosure, the method further includes:

[0166] Use the global storage path to store video rendering data.

[0167] In a second aspect, according to one or more embodiments of the present disclosure, a video processing device is provided, including:

[0168] A split unit is used to split the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process;

[0169] A rendering unit, used for synthesizing and rendering the material track segments in a target time segment among the multiple time segments on the editing track, generating and caching video rendering data of the target time segment;

[0170] The display unit is used for responding to receiving the video editing preview instruction, if the preview position is located in the target time period, reading and displaying the rendering data of the preview position from the video rendering data buffered in the target time period.

[0171] According to one or more embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period and the end point of the previous time period of the time period;

[0172] The end point of each material track segment is the end point of a time period and the starting point of the next time period after the time period.

[0173] According to one or more embodiments of the present disclosure, the device further includes an identification configuration unit. The identification configuration unit is used to:

[0174] For each target time period, a first identifier corresponding to the target time period is generated according to the picture data of the material track segment corresponding to the target time period; and a first identifier is set for the cached video rendering data corresponding to the target time period.

[0175] According to one or more embodiments of the present disclosure, the display unit is further configured to:

[0176] Determine a first identifier corresponding to the target time period;

[0177] The rendering data corresponding to the preset preview position is determined from the video rendering data corresponding to the first identifier read from the cache.

[0178] According to one or more embodiments of the present disclosure, the video processing device further includes a mapping unit, and the mapping unit is used to:

[0179] For the video rendering data of the target time period, a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period is established; and the display unit is further used for:

[0180] Determining, from a cache, video rendering data corresponding to the target time period according to a first identifier of the target time period;

[0181] The rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the mapping relationship.

[0182] According to one or more embodiments of the present disclosure, the mapping unit is further configured to:

[0183] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0184] A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated from the frame information.

[0185] According to one or more embodiments of the present disclosure, the mapping unit is further configured to:

[0186] For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0187] According to one or more embodiments of the present disclosure, the segmentation unit is further configured to:

[0188] In response to receiving a re-editing operation performed on the material track segment, splitting the timeline according to information of the material track segment on the editing track;

[0189] Rendering units are further used to:

[0190] A new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0191] The video processing device further includes a storage unit, which is used for:

[0192] Use the global storage path to store video rendering data.

[0193] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0194] Memory stores computer-executable instructions;

[0195] At least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the first aspect as well as various video processing methods that may be involved in the first aspect.

[0196] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the first aspect and various video processing methods that may be involved in the first aspect are implemented.

[0197] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the above first aspect and various video processing methods that may be involved in the first aspect.

[0198] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0199] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0200] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video processing method, comprising: During the video editing process, the timeline used for video editing is divided into multiple time periods according to the material track clips on the editing track; Performing synthesis rendering on the material track segments on the editing track that are located in a target time segment among the multiple time segments, generating and caching video rendering data of the target time segment; In response to receiving the video editing preview instruction, if the preview position is within the target time period, the rendering data of the preview position is read from the video rendering data cached in the target time period and displayed.

2. The method according to claim 1, characterized in that The starting point of each of the material track segments is the starting point of a time period and the end point of the previous time period of the time period; The end point of each material track segment is the end point of a time period and the starting point of the next time period after the time period.

3. The method according to claim 1, characterized in that: The method further comprises: For each target time period, a first identifier corresponding to the target time period is generated according to the picture data of the material track segment corresponding to the target time period; and the first identifier is set for the cached video rendering data corresponding to the target time period.

4. The method according to claim 3, characterized in that: In response to receiving the video editing preview instruction, if the preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period, comprises: Determine a first identifier corresponding to the target time period; The rendering data corresponding to the preview position is determined from the video rendering data corresponding to the first identifier read from the cache.

5. The method according to claim 4, characterized in that The method further comprises: For the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and The determining, from the video rendering data corresponding to the first identifier read from the cache, rendering data corresponding to the preview position includes: Determining, from a cache, video rendering data corresponding to the target time period according to a first identifier of the target time period; The rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the mapping relationship.

6. The method according to claim 5, characterized in that The step of establishing a mapping relationship between the rendering data of each time period and the frame information of the material track segment in the time period includes: Determine the frame information of the material track segment corresponding to the target time period in the corresponding material; A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated by the frame information.

7. The method according to claim 6, characterized in that The determining of frame information of the material track segment corresponding to the target time period in the corresponding material includes: For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

8. The method according to claim 1, characterized in that: The method further comprises: In response to receiving a re-editing operation performed on the material track segment, segmenting the timeline according to information of the material track segment on the editing track; A new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: The video rendering data is stored using a global storage path.

10. A video processing device, comprising: A split unit is used to split the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process; A rendering unit, configured to perform synthesis rendering on the material track segments on the editing track within a target time segment among the multiple time segments, generate video rendering data of the target time segment, and cache the data; The display unit is used for, in response to receiving a video editing preview instruction, reading and displaying rendering data of the preview position from the video rendering data cached in the target time period if the preview position is within the target time period.

11. An electronic device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the video processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video processing method according to any one of claims 1 to 9 is implemented.

Images