Video processing method and device, and electronic device

Abstract

The method comprises the following steps: in a video editing process, a time axis for video editing is divided into a plurality of time periods according to material track segments on an edit track; a material track segment on the edit track located in a target time period in the plurality of time periods is synthesized and rendered to generate video rendering data of the target time period and to cache; in response to receiving a video editing preview instruction, if a preview position is located in the target time period, rendering data of the preview position is read from the cached video rendering data of the target time period and is displayed. The phenomenon of lag caused by real-time rendering when previewing an editing effect in a video editing process can be reduced, and thus the user experience in the video editing process can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly, to a video processing method and device and electronic device. BACKGROUND

[0002] Video editing generally includes a process of adding special effects, scenes to the already recorded video, or mixing video clips from different video sources to obtain a new video.

[0003] The video editing and editing effect preview process described above requires a large amount of system resources. When the electronic device displays the editing effect during video editing, the phenomenon of frame freezing usually occurs. Users usually preview the editing effect frequently during the video editing process, and the problem of frame freezing is encountered when previewing the editing effect multiple times. Therefore, the user experience of video editing is poor. SUMMARY

[0004] Embodiments of the present disclosure provide a video processing method and device and electronic device to overcome the problem of poor experience caused by frame freezing when previewing the editing effect during video editing in the related art.

[0005] In a first aspect, the embodiments of the present disclosure provide an image processing method, which includes: in a video editing process, dividing a time axis for video editing into a plurality of time periods according to material track segments on an edit track; performing synthetic rendering on material track segments on the edit track located in a target time period in the plurality of time periods to generate video rendering data of the target time period and cache; in response to receiving a video editing preview instruction, if a preview position is located in the target time period, reading rendering data of the preview position in the cached video rendering data of the target time period and displaying.

[0006] In a second aspect, the embodiments of the present disclosure provide an image processing device, which includes: a dividing unit configured to divide a time axis for video editing into a plurality of time periods according to material track segments on an edit track in a video editing process; a rendering unit configured to perform synthetic rendering on material track segments on the edit track located in a target time period in the plurality of time periods to generate video rendering data of the target time period and cache; and a display unit configured to, in response to receiving a video editing preview instruction, if a preview position is located in the target time period, read rendering data of the preview position in the cached video rendering data of the target time period and display.

[0007] In a third aspect, an electronic device is provided, and the electronic device includes a processor and a memory. The memory stores computer-executable instructions. The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing method according to the first aspect and various possible implementations of the first aspect.

[0008] In a fourth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the video processing method according to the first aspect and various possible implementations of the first aspect is implemented.

[0009] In a fifth aspect, a computer program product is provided, and the computer program product includes a computer program. When a processor executes the computer program, the video processing method according to the first aspect and various possible implementations of the first aspect is implemented.

[0010] The video processing method, device and electronic device provided in the embodiments can divide a time axis for video editing into a plurality of time periods according to material track segments on an edit track in a video editing process, perform synthetic rendering on material track segments on the edit track in a target time period of the plurality of time periods, generate video rendering data of the target time period and cache the video rendering data, and in response to receiving a video editing preview instruction, read rendering data of a preview position from the video rendering data of the target time period that has been cached and display the rendering data, because the time axis for video editing is divided into a plurality of time periods according to material track segments on the edit track, and synthetic rendering is performed on material track segments in a target time period, a preview video can be generated according to video rendering data corresponding to different target time periods when a video editing effect is previewed, which can reduce the lag caused by real-time rendering when a video editing effect is previewed in the video editing process, and thus the user experience in the video editing process can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0012] Figure 1 A video editing interface schematic diagram provided by the embodiments of the present disclosure;

[0013] Figure 2 A flowchart of a video processing method provided by the embodiments of the present disclosureFigure 1

[0014] Figure 3 A schematic diagram of time axis segmentation provided by an embodiment of the present disclosure;

[0015] Figure 4 A flowchart of a video processing method provided by an embodiment of the present disclosure Figure 2

[0016] Figure 5 A scene schematic diagram provided by the present disclosure;

[0017] Figure 6 A scene schematic diagram provided by the present disclosure;

[0018] Figure 7 A structural block diagram of a video processing device provided by an embodiment of the present disclosure;

[0019] Figure 8 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0021] Reference is made to Figure 1 , which shows a schematic diagram of a video editing. The schematic diagram is a process of mixing two video materials. The video editing mixes video material 10 and video material 20. The video editing interface can include track 11 and track 12 below time axis 13. Track 11 includes material track segments composed of multiple frames selected from video material 10. Track 12 can display material track segments composed of multiple frames selected from video material 20.

[0022] The user drags the material track segment at track 11 and drags the material track segment at video frame area 12, so as to select the frames at which video material 10 and video material 20 start to be mixed. After multiple mixing frames of the two are determined, effect preview can be performed to browse the editing effect.

[0023] Figure 1 ​​Only an example of video editing, in practice, multiple video materials can be mixed, or text, special effect graphics, etc. can be added to a video. During video editing, users usually preview the editing effect in order to view the editing effect. If the electronic device used by the user for video editing has low resource configuration, or the material used for video editing is large itself, when previewing the editing effect, since each frame needs to be rendered and a preview picture is generated according to the rendering result, since each frame picture corresponding to the rendering editing result needs more system resources, the preview editing result display will appear a lag phenomenon, resulting in poor user experience.

[0024] In order to improve the above problems, the scheme provided by the present disclosure divides the time axis into multiple time periods according to the material track segment on the editing track, generates video rendering data for a target time period in the multiple time periods and caches it, and when previewing, the video rendering data of the target time period can be used to generate a preview picture. In addition, the video rendering data processed in the above manner can be reused, which can reduce the resources required when previewing the video editing effect, thereby reducing the lag phenomenon.

[0025] Please refer to Figure 2 , Figure 2 A flowchart of a video processing method provided by an embodiment of the present disclosure Figure 1 The execution subject of the video processing method in the present disclosure can be a terminal device, and the method comprises:

[0026] S201: In the video editing process, the time axis for video editing is divided into multiple time periods according to the material track segment on the editing track.

[0027] In the above terminal device, various video editing applications can be run. In the video editing application, the video material to be edited can be imported, and then the editing operation is performed on the video material.

[0028] The above-mentioned video material to be edited can include video materials obtained from various methods, including but not limited to: video materials recorded using a video recording device, video materials obtained from other electronic devices based on authorization, etc. The above-mentioned video material can also include one of the following: a preset special effect, a preset background, etc. Here, the video material to be edited is only explained by way of example, and all video materials are not exhausted. The above examples do not limit the protection scope of the present disclosure scheme.

[0029] Generally in a video editing application, a timeline can be displayed in the video editing interface. The timeline generally refers to a list of thumbnails appearing under the video preview and moving along with the video playback when editing the video. Through the timeline, the content of the video at a certain time point can be controlled. In addition, by using the timeline and network technology, multiple events can be connected in time sequence to form a relatively complete recording system.

[0030] For a video material in the video editing material, the video material is transcoded and displayed under the timeline. A plurality of video frames corresponding to the video material are displayed in the order of appearance time. As shown in Figure 1 For video 1, a plurality of video frames of video material 10 can be displayed under timeline 13.

[0031] In the video editing process, at least one material track segment can be included in the editing track.

[0032] The material track segment includes a text material track segment, a graphic effect material track segment, or a video material track segment.

[0033] The above-mentioned timeline can be divided according to the material track segment on the editing track. A plurality of time periods are obtained.

[0034] As an implementation manner, the timeline can be divided according to the start point and end point of the material track segment as the division points of the timeline. For a plurality of material track segments, the start point and end point of each material track segment can be used as the division points of the timeline. For the material track segments in different tracks, the start point and end point of the material track segment in each track can be used as the division points of the timeline.

[0035] The timeline can be divided by the above-mentioned division points to obtain a plurality of time periods.

[0036] In this way, the start point of each material track segment is the start point of a time period, and is also the end point of the previous time period of the time period;

[0037] In the above-mentioned time period division manner, the end point of each material track segment is the end point of a time period, and is also the start point of the next time period of the time period.

[0038] In each time period, there is no more than one material track segment data in the same track of the time period.

[0039] Please refer to Figure 3 , Figure 3A segmentation diagram of a time axis is shown. Different video materials can correspond to different tracks, for example, video material 1 and video material 2 can correspond to track 1 and track 2 respectively. It is assumed that the first frame of track material segment 1 appears at time t0 on the time axis (i.e., t0 is the starting point of track material segment 1) on track 1, the first frame corresponding to material track segment 2 in video material 2 appears at t1 (t1 is the starting point of track material segment 2), material track segment 1 ends at time t2 (t2 is the end point of track material segment 1); material track segment 2 ends at time t3 (t3 is the end point of track material segment 2). In this way, since the time axis can be segmented by t0, t1, t2, and t3 as segmentation points, time period 1 corresponding to t0-t1, time period 2 corresponding to t1-t2, and time period 3 corresponding to t2-t3 are obtained. t1 is the end point of time period 1 and the starting point of time period 2. t2 is the starting point of time period 3 and the end point of time period 2.

[0040] The plurality of time periods obtained by the above segmentation method have good continuity of the material track segments in each track. In addition, each time period obtained by the above segmentation method does not include multiple material track segments, so that the amount of calculation required for generating video rendering data for each time period is small, which helps to quickly generate video rendering data for each time period.

[0041] S202: Synthesis rendering is performed on the material track segments in the target time period in the plurality of time periods on the edit track, video rendering data of the target time period is generated and cached.

[0042] After obtaining the plurality of time periods in step S101, a target time period in the plurality of time periods can be determined. The target time period here can be a time period corresponding to a larger number of material track segments in the plurality of time periods, or a time period corresponding to a material track segment with more changes in picture information.

[0043] For the target time period, the corresponding material track segments in the target time period can be determined, and then a synthesis rendering task is generated. The synthesis rendering task is then executed. Specifically, the corresponding material track segments in the target time period can be synthesized according to a preset synthesis rule, and then rendered to obtain video rendering data corresponding to the target time period.

[0044] For example, for each material track segment in the target time period, the video frames corresponding to the material track segment are synthesized in units of frames, and then rendered. Or the video frames and graphic frames in the material track segment are synthesized in units of frames, and then rendered. Therefore, the video rendering data corresponding to the target time period can include the synthesized rendering data corresponding to the multiple frames belonging to the target time period.

[0045] In practice, the rendering control management object provided by the video editing application can be invoked to perform the synthesis rendering task.

[0046] For the synthesis rendering tasks corresponding to the multiple target time periods, the tasks can be executed in series or in parallel.

[0047] For each target time period, the video rendering data corresponding to the target time period can be cached.

[0048] In some embodiments, the video processing method further includes caching the video rendering data using a global storage path.

[0049] In these embodiments, the global storage path can be a storage path other than the draft of the video material being edited.

[0050] After setting the global storage path, the video rendering data can be associated with the draft of the video material. In this way, when the draft name is modified, the video rendering data is not affected by the change in the draft name.

[0051] S203: In response to receiving the video editing preview instruction, if the preview position is located in the target time period, read the rendering data of the preview position from the cached video rendering data of the target time period and display.

[0052] During the editing process, the user often needs to browse the editing effect. The user can click the preview control during the editing operation to preview the editing effect, thereby sending the preview instruction to the execution subject.

[0053] After receiving the preview instruction, the preview position indicated by the preview instruction can be determined. If the preview position indicated by the preview instruction is located in the target time period, the rendering data corresponding to the preview position is read from the cached video rendering data of the target time period and displayed.

[0054] In this embodiment, in a video editing process, a timeline for video editing is divided into multiple time periods according to material track segments on an edit track; a target time period in the multiple time periods is determined according to a user's selection operation; and material track segments in the target time period are rendered to generate video rendering data of the target time period and cache the video rendering data. In response to receiving a video editing preview instruction, if a preview position is located in the target time period, rendering data of the preview position is read from the cached video rendering data of the target time period and displayed. Since the timeline is divided according to the material track segments on the edit track, the material track segments in the target time period are rendered, and the preview picture is generated according to the video rendering data corresponding to the target time period when the video editing effect is previewed, the lag phenomenon caused by real-time rendering when the editing effect is previewed in the video editing process can be reduced, and the user experience in the video editing process can be improved.

[0055] Please refer to Figure 4 , which is a flowchart of a video processing method provided by an embodiment of the present disclosure Figure 2 The execution subject of the video processing method in the present disclosure can be a terminal device, and the method comprises the following steps.

[0056] S401: In a video editing process, a timeline for video editing is divided into multiple time periods according to material track segments on an edit track.

[0057] S402: Material track segments in a target time period in the multiple time periods on the edit track are rendered to generate video rendering data of the target time period and cache the video rendering data.

[0058] The specific implementation of steps S401-S402 can refer to the related description of the embodiment shown in Figure 2 Here, no further description is given.

[0059] S403: For each target time period, a first identifier corresponding to the target time period is generated according to picture data of a material track segment corresponding to the target time period; and the first identifier is set for the cached video rendering data corresponding to the target time period.

[0060] The target time period can be a time period in the multiple time periods obtained in step S401. Illustratively, the target time period can be a time period determined from the multiple time periods according to a user's selection operation. It can also be a time period determined according to the size of the picture information transformation in each time period.

[0061] The first identifier here can be a first identifier obtained using various digital signature algorithms.

[0062] Illustratively, for each target time period,Figure 3 In the time period 2 (target time period) in the time period 1, the picture data in the material track segment 1 and the material track segment 2 corresponding to the time period 2 can be used to determine the first identification of the time period 2 by using the digital signature algorithm. The first identification obtained by using the digital signature algorithm is usually a unique identification.

[0063] Illustratively, the digital signature algorithm can be the Message Digest Algorithm (MD5).

[0064] The first identification has the properties of uniqueness and irreversibility. When the video editing data input to the digital signature algorithm is changed, for example, the video frames are added or reduced, or special effects or backgrounds are added, etc., the first identification obtained by the digital signature algorithm will change.

[0065] By the above steps, for the target time period in the plurality of time periods, the track material segment in the target time period can be synthesized and rendered to obtain video rendering data. The first identification is cached and set for the video rendering data of the target time period. Since the first identification has the property of uniqueness, the video rendering data can be accurately found in the cache by the first identification corresponding to the target time period.

[0066] For example, for the time period 2 in the time period 1, Figure 2 In the time period 2, the video material 1 and the video material 2 corresponding to the time period 2 can be pre-rendered by synthesizing each frame of picture data of the material track segment 1 of the video material 1 and each frame of picture data of the material track segment 2 of the video material 2 to obtain the video rendering data of the time period 2. The first identification of the time period 2 can be determined by using the digital signature algorithm on the plurality of frames of picture data in the material track segment 1 and the plurality of frames of picture data in the material track segment 2 in the time period 2, and the first identification is set for the cached video rendering data of the time period 2.

[0067] Thus, for the same time period, as long as the picture data of the material track segment of each track in the time period is not changed, the first identification calculated by using the same digital signature algorithm on the picture data in the material track segment in the time period is the same. Based on this, when the editing effect is previewed, the video rendering data corresponding to each target time period can be accurately found according to the first identification.

[0068] In addition, by adding the first identification to the identification of the video rendering data of the target time period, multiple synthesis rendering operations on the same material track segment in the same time period can be avoided.

[0069] S404: In response to receiving the video editing preview instruction, if the preview position is located in the target time period, the first identification corresponding to the target time period is determined.

[0070] S405: Determine the rendering data corresponding to the preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0071] Upon receiving the aforementioned video editing preview instruction, the time period within which the instruction falls can be determined. For example, a first identifier for that time period can be generated based on the image data of each editable clip within that time period using the same algorithm used to generate the first identifier. This first identifier is then used for matching within the cache. If a match is successful, it indicates that the time period is the target time period. The corresponding video rendering data is then retrieved from the cache.

[0072] Then, the rendering data corresponding to the preview position is extracted from the video rendering data corresponding to the target time period.

[0073] In this embodiment, compared to Figure 2 The solution in the illustrated embodiment generates a first identifier related to the video clip data of the source material track for the target time period and caches the video rendering data for the target time period according to the first identifier. Therefore, upon receiving a preview command, the video rendering data for the target time period can be accurately and quickly retrieved from the cache based on the first identifier, and the rendering data corresponding to the preview position can be extracted. This improves the accuracy of the preview effect displayed by the preview command.

[0074] In some embodiments, the video processing method further includes: for video rendering data of a target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segments in the target time period; and step S405 further includes the following steps:

[0075] First, based on the first identifier of the target time period, determine the video rendering data corresponding to the target time period from the cache.

[0076] Secondly, based on the above mapping relationship, the rendering data of the preview position is determined from the video rendering data corresponding to the target time period.

[0077] Here, establishing a mapping relationship between video rendering data and frame information of media track segments clarifies which frames in the media correspond to a given set of video rendering data. This facilitates the process of first locating the video rendering data corresponding to a target time segment based on the first identifier when a preview command indicates such a segment. Then, by extracting the rendering data corresponding to the preview position indicated by the preview command from the video rendering data based on the frame information, the editing effect corresponding to the preview position can be accurately and quickly displayed.

[0078] In some embodiments, step S404 above includes the following steps:

[0079] determine frame information of the material track segment corresponding to the target time period in the corresponding material;

[0080] establish a mapping relationship between the video rendering data of the target time period and the frame information corresponding to the target time period by using the second identifier of the video rendering data of the target time period generated by the frame information.

[0081] Illustratively, for the video rendering data in time period 1 in Figure 3 The frame information of time period 1 can be the frame information "1-5 frame video images" of the material track segment in video material 1. The second identifier can be "video material 1 video rendering data corresponding frame (1-5.mp4)". The second identifier of the target time period is generated by including the frame information, and the second identifier is set as part of the identifier of the video rendering data of the target time period. Thus, a mapping relationship between the video rendering data of the target time period and the frame information corresponding to the target time period is established.

[0082] In these embodiments, the second identifier generated by the frame information can be set for the video rendering data. Then the video frame corresponding to the rendering data in the target time period can be quickly determined according to the second identifier, so that the rendering data corresponding to the preview position can be quickly determined from the cache according to the second identifier, and the speed of preview editing effect can be further improved.

[0083] In some embodiments, the above-mentioned determination of the frame information of the material track segment corresponding to the target time period in the corresponding material includes:

[0084] For the multi-track material track segment in the target time period, the frame information of the material track segment corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment appearing latest on the time axis in the corresponding material.

[0085] In these embodiments, the above-mentioned editing track can include multiple tracks, and the material can include multiple materials.

[0086] The following will be described in combination with Figure 5 Figure 5 ​An application scenario provided by the present disclosure is shown in the following figure. During the editing process, the target time period D on the timeline corresponds to the material track segment 3 extracted from the video material 1, which is the segment from the 100th frame to the 200th frame of the video material 1. The material track segment 4 is extracted from the video material 2. The material track segment of the video material 2 corresponding to the target time period D is the segment from the 75th frame to the 175th frame of the video material 2. The user performs a mixing editing operation on the material track segment 3 and the material track segment 4. On the editing track, the appearance time of the material track segment 4 is later than that of the material track segment 3. The first frame of the video material 4 can be used as the starting position of the frame information of the target time period D. This is because the video material 1 and the video material 2 are mixed from the first frame of the video material 2 at most.

[0087] In addition, the first identifier E can be generated according to the picture data from the 100th frame to the 200th frame of the video material 1 and the 75th frame to the 175th frame of the video material 2. The second identifier can be generated according to the frame information of the 75th frame to the 175th frame of the video material 2. The video rendering data of the material track segment 3 and the material track segment 4 corresponding to the above-mentioned target time period D can be generated, and the first identifier and the second identifier can be set for the video rendering data. The mapping relationship between the frame information 75-175 and the video rendering data is established by using the second identifier. The second identifier can be, for example, “75-175.mp4”.

[0088] In these embodiments, for the editing scenario of the multi-track video material, for the target time period, the frame mapping information of the video rendering data of the target time period and the track material segment is determined according to the frame information of the track material segment in the corresponding material that appears on the timeline at the latest, so that the video rendering data generated by the historical editing operation can be reused when the user performs subsequent editing operation on the same video material. Therefore, the system resources can be saved.

[0089] In some embodiments, the video processing method further includes the following steps:

[0090] First, in response to receiving the re-editing operation performed on the material track segment, the timeline is segmented according to the information of the material track segment on the editing track.

[0091] Secondly, a new target time period that does not overlap with the historical target time period is determined from the segmentation result, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0092] In these embodiments, the video editing usually needs multiple editing operations on the video material to finally generate a video editing result that satisfies the user.

[0093] After the video rendering data is generated according to the previous editing operation and previewed, the user can continue to perform the editing operation on the video editing material.

[0094] After receiving the re-editing operation of the user on the video editing material, the time axis is segmented again according to the segmentation process in step S401, and a plurality of segmentation results can be obtained.

[0095] For each segmentation result, it can be judged whether the segmentation result coincides with the historical target time period. If yes, no new target time period and corresponding video rendering data are generated. If part of the segmentation result coincides with the historical target time period, a new target time period can be generated according to the non-coincidence part. If the segmentation result has no part coinciding with the historical target time period, the segmentation result can be used as a new target time period.

[0096] For the new target time period, video rendering data is generated, and the video rendering data is cached. When caching, the first identifier and the second identifier are set for the video rendering data. The first identifier is generated from the picture data of the new target time period, and the second identifier is generated from the frame information of the material track segment corresponding to the new target time period in the corresponding material.

[0097] When receiving a preview instruction for the re-editing operation, for the target time period coinciding with the historical target time period, the video rendering data corresponding to the historical target time period can be reused for preview presentation.

[0098] In combination Figure 5 and Figure 6 , Figure 6 An application scenario provided by the present disclosure is shown. For Figure 5 In the application scenario, the user performs a mixed editing operation on the 100th frame to the 200th frame of the video material 1 and the 75th frame to the 175th frame of the video material 2. In the subsequent editing operation, the video material 1 is moved to the left on the time axis, and the number of frames mixed with the video material 2 is increased. As shown in Figure 6 , after the re-editing operation of the user, the 75th frame to the 200th frame of the video material 1 is mixed with the 50th frame to the 175th frame of the video material 2. That is, the video material 1 corresponds to the material track segment 3', and the video material 2 corresponds to the material track segment 4, as shown in Figure 6 .

[0099] For Figure 6According to the editing operation shown, the new target time period F corresponding to the 75th frame to the 99th frame of the video material 1 can be generated in the manner described above. For the new target time period F, the first identifier P is generated according to the picture data of the 75th frame to the 99th frame of the video material 1 and the 50th frame to the 74th frame of the video material 2. The second identifier of the new target time period is generated according to the frame information of the track material segment 4 in the video material 2 corresponding to the new target time period F. The video rendering data of the new target time period is generated, and the mapping relationship between the video rendering data and the frame information corresponding to the new target time period is established by using the second identifier. The video rendering data information is cached, and the identifier including the first identifier and the second identifier is used for the video rendering data.

[0100] For the 100th frame to the 200th frame of the video material 1 and the 75th frame to the 175th frame of the video material 2, the video rendering data corresponding to the original target time period D can be reused. In this way, when previewing the editing operation shown in Figure 6 , the video rendering data corresponding to the target time period D generated by the historical editing operation shown can be directly used for previewing. When the preview position is located in the target time period F, the newly generated video rendering data can also be used for previewing. Figure 5

[0101] In these embodiments, for multiple editing operations, the video rendering data corresponding to the original historical target time period can be reused, thereby reducing the resources required for rendering for each editing operation, and thereby enabling the user to display the preview picture generated according to the video rendering data more smoothly each time.

[0102] The video processing method corresponding to the above embodiments, Figure 7 The structure block diagram of the video processing apparatus provided by the embodiments of the present disclosure is shown. For ease of illustration, only the parts related to the embodiments of the present disclosure are shown. Referring to Figure 7 , the apparatus 70 includes a segmentation unit 701, a rendering unit 702, and a display unit 703. Wherein:

[0103] The segmentation unit 701 is configured to divide the time axis for video editing into multiple time periods according to the material track segments on the editing track during video editing;

[0104] The rendering unit 702 is configured to perform synthetic rendering on the material track segments on the editing track located in the target time period in the multiple time periods, generate video rendering data of the target time period and cache;

[0105] The display unit 703 is configured to, in response to receiving a video editing preview instruction, read the rendering data of the preview position in the video rendering data cached in the target time period and display if the preview position is located in the target time period.​

[0106] In some embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period, and is also the ending point of a previous time period of the time period;

[0107] The ending point of each material track segment is the ending point of a time period, and is also the starting point of a next time period of the time period.

[0108] In some embodiments of the present disclosure, the device 70 further comprises an identifier configuration unit (not shown in the figure). The identifier configuration unit is configured to:

[0109] For each target time period, generate a first identifier corresponding to the target time period according to the picture data of the material track segment corresponding to the target time period; and set the first identifier for the cached video rendering data corresponding to the target time period.

[0110] In some embodiments of the present disclosure, the display unit 703 is further configured to:

[0111] Determine the first identifier corresponding to the target time period;

[0112] Determine the rendering data corresponding to the preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0113] In some embodiments of the present disclosure, the device 70 further comprises a mapping unit (not shown in the figure), and the mapping unit is configured to:

[0114] For the video rendering data of the target time period, establish a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and the display unit 703 is further configured to:

[0115] According to the first identifier of the target time period, determine the video rendering data corresponding to the target time period from the cache;

[0116] According to the mapping relationship, determine the rendering data of the preview position from the video rendering data corresponding to the target time period.

[0117] In some embodiments of the present disclosure, the mapping unit is further configured to:

[0118] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0119] Establish a mapping relationship between the video rendering data of the target time period and the frame information corresponding to the target time period by using the second identifier of the video rendering data of the target time period generated by the frame information.

[0120] In some embodiments of the present disclosure, the mapping unit is further configured to:

[0121] For the multi-track material track segment in the target time period, according to the frame information of the material track segment corresponding to the target time period in the corresponding material, the frame information of the material track segment corresponding to the target time period in the corresponding material is determined.

[0122] In some embodiments of the present disclosure, the splitting unit is further configured to:

[0123] In response to receiving a re-edit operation performed on the material track segment, the timeline is split according to the information of the material track segment on the edit track;

[0124] The rendering unit is further configured to:

[0125] From the splitting result, a new target time period that does not overlap with the historical target time period is determined, and for the material track segment corresponding to the new target time period, video rendering data is generated and cached.

[0126] In some embodiments of the present disclosure, the device 70 further includes a storage unit (not shown in the figure), and the storage unit is configured to:

[0127] The video rendering data is stored using a global storage path.

[0128] To implement the above-mentioned embodiments, the present disclosure further provides an electronic device.

[0129] Reference Figure 8 , which shows a structural schematic diagram of an electronic device 800 suitable for implementing embodiments of the present disclosure. The electronic device 800 can be various terminal devices, servers. Among them, the terminal device can include an extended reality device. The extended reality device may, for example, include various wearable devices, such as a head-mounted extended reality device, etc. Figure 8 The electronic device shown is only an example and should not impose any limitations on the functions and use range of the embodiments of the present disclosure.

[0130] As Figure 8 shown, the electronic device 800 can include a processing device (such as a central processor, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (Read Only Memory, ROM for short) 802 or loaded from a storage device 808 into a random access memory (Random Access Memory, RAM for short) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0131] In general, the following devices can be connected to the I / O interface 805: input devices 806, including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 807, including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 808, including, for example, a magnetic tape, a hard disk, and the like; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or via a wire with other devices to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it is understood that all of the shown devices are not required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0132] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing devices 801, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0133] It should be noted that the computer-readable medium in the above disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0134] The computer-readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device.

[0135] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0136] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0137] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0138] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the dividing unit can also be described as a unit that divides a time axis for video editing into multiple time periods according to a material track segment on an edit track in a video editing process.

[0139] The functions described above in the detailed description of embodiments of the present disclosure can be performed by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0140] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] In a first aspect, according to one or more embodiments of the present disclosure, a video processing method is provided, comprising:

[0142] In a video editing process, a timeline for video editing is divided into a plurality of time periods according to material track segments on an edit track;

[0143] The material track segments on the edit track in the target time period are synthesized and rendered to generate video rendering data of the target time segment and cache the video rendering data;

[0144] In response to receiving a video editing preview instruction, if the preview position is located in the target time period, the rendering data of the preview position is read from the cached video rendering data of the target time period and displayed.

[0145] According to one or more embodiments of the present disclosure,

[0146] The starting point of each material track segment is the starting point of a time period, and is also the ending point of the previous time period of the time period;

[0147] The ending point of each material track segment is the ending point of a time period, and is also the starting point of the next time period of the time period.

[0148] According to one or more embodiments of the present disclosure, the method further comprises: for each target time period, generating a first identifier corresponding to the target time period according to the picture data of the material track segment corresponding to the target time period; and setting the first identifier for the cached video rendering data corresponding to the target time period.

[0149] According to one or more embodiments of the present disclosure, in response to receiving the video editing preview instruction, if the preview position is located in the target time period, reading and displaying the rendering data of the preview position in the video rendering data of the target time period that has been cached, comprising:

[0150] determining the first identifier corresponding to the target time period;

[0151] determining the rendering data corresponding to the preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0152] According to one or more embodiments of the present disclosure, the method further comprises:

[0153] for the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and

[0154] determining the rendering data corresponding to the preview position from the video rendering data corresponding to the first identifier read from the cache.

[0155] According to the first identifier of the target time period, determining the video rendering data corresponding to the target time period from the cache;

[0156] According to the mapping relationship, determining the rendering data of the preview position from the video rendering data corresponding to the target time period.

[0157] According to one or more embodiments of the present disclosure, for the rendering data of each time period, establishing a mapping relationship between the rendering data and the frame information of the material track segment in the time period, comprising:

[0158] determining the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0159] using the second identifier of the video rendering data of the target time period generated by the frame information, establishing a mapping relationship between the video rendering data and the frame information corresponding to the target time period.

[0160] According to one or more embodiments of the present disclosure, determining the frame information of the material track segment corresponding to the target time period in the corresponding material, comprising:

[0161] for the multi-track material track segment in the target time period, according to the frame information of the material track segment corresponding to the latest appearing material track segment in the time axis in the corresponding material, determining the frame information of the material track segment corresponding to the target time period in the corresponding material.

[0162] According to one or more embodiments of the present disclosure, the method further comprises:

[0163] In response to receiving a re-edit operation performed on the material track segment, the timeline is segmented according to information of the material track segment on the edit track;

[0164] A new target time period that has no overlap with the historical target time period is determined from the segmentation result, and video rendering data is generated for the material track segment corresponding to the new target time period and is cached.

[0165] According to one or more embodiments of the present disclosure, the method further includes:

[0166] The video rendering data is stored using a global storage path.

[0167] In a second aspect, according to one or more embodiments of the present disclosure, a video processing apparatus is provided, including:

[0168] A segmentation unit is configured to segment a timeline for video editing into a plurality of time periods according to material track segments on an edit track during a video editing process;

[0169] A rendering unit is configured to perform a composite rendering on a material track segment on the edit track located within a target time period of the plurality of time periods, generate video rendering data of the target time period and cache the video rendering data.

[0170] A display unit is configured to, in response to receiving a video editing preview instruction, if a preview position is located within the target time period, read rendering data of the preview position from the cached video rendering data of the target time period and display the rendering data.

[0171] According to one or more embodiments of the present disclosure, a start point of each material track segment is a start point of a time period and a terminal point of a previous time period of the time period;

[0172] A terminal point of each material track segment is a terminal point of a time period and a start point of a next time period of the time period.

[0173] According to one or more embodiments of the present disclosure, the apparatus further includes an identification configuration unit. The identification configuration unit is configured to:

[0174] For each target time period, a first identification corresponding to the target time period is generated according to picture data of a material track segment corresponding to the target time period; and the first identification is set for the cached video rendering data corresponding to the target time period.

[0175] According to one or more embodiments of the present disclosure, the display unit is further configured to:

[0176] Determine the first identification corresponding to the target time period;

[0177] The video rendering data corresponding to the first identifier read from the cache is determined.

[0178] According to one or more embodiments of the present disclosure, the video processing apparatus further comprises a mapping unit, configured to:

[0179] For the video rendering data of the target time period, a mapping relationship between the video rendering data and frame information of the material track segment in the target time period is established; and the display unit is further configured to:

[0180] According to the first identifier of the target time period, the video rendering data corresponding to the target time period is determined from the cache;

[0181] According to the mapping relationship, the rendering data of the preview position is determined from the video rendering data corresponding to the target time period.

[0182] According to one or more embodiments of the present disclosure, the mapping unit is further configured to:

[0183] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material;

[0184] Using the second identifier of the video rendering data of the target time period generated by the frame information, a mapping relationship between the video rendering data and the frame information corresponding to the target time period is established.

[0185] According to one or more embodiments of the present disclosure, the mapping unit is further configured to:

[0186] For the multi-track material track segment in the target time period, according to the frame information of the material track segment appearing latest on the time axis in the corresponding material, the frame information of the material track segment corresponding to the target time period in the corresponding material is determined.

[0187] According to one or more embodiments of the present disclosure, the segmentation unit is further configured to:

[0188] In response to receiving a re-editing operation performed on the material track segment, the time axis is segmented according to the information of the material track segment on the edit track;

[0189] The rendering unit is further configured to:

[0190] From the segmentation result, a new target time period that does not overlap with the historical target time period is determined, and for the material track segment corresponding to the new target time period, video rendering data is generated and cached.

[0191] The video processing apparatus further comprises a storage unit, configured to:

[0192] The video rendering data is stored using a global storage path.

[0193] In a third aspect, an electronic device is provided, including: at least one processor and a memory, according to one or more embodiments of the present disclosure;

[0194] The memory stores computer-executable instructions;

[0195] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video processing method as described in the first aspect above and various possible aspects related to the first aspect.

[0196] In a fourth aspect, a computer-readable storage medium is provided, storing computer-executable instructions, when a processor executes the computer-executable instructions, the video processing method as described in the first aspect above and various possible aspects related to the first aspect is implemented, according to one or more embodiments of the present disclosure.

[0197] In a fifth aspect, a computer program product is provided, including a computer program, when a processor executes the computer program, the video processing method as described in the first aspect above and various possible aspects related to the first aspect is implemented, according to one or more embodiments of the present disclosure.

[0198] The above description is merely exemplary of the present disclosure and the application of the principles thereof and the scope of what may be claimed is not to be construed as limited to the preferred embodiments described herein. The description of the preferred embodiments is not intended to limit the scope of the present disclosure and the application of the principles thereof to the preferred embodiments described herein. The scope of what may be claimed is intended to be limited only by the claims so as to be construed as including all equivalents to the preferred embodiments described herein.

[0199] Furthermore, while operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order depicted and described. Many of the operations can in fact be performed in parallel, concurrently, or in any order practical. In addition, while a particular implementation has been described, this should not be understood as requiring or implying that all or even any of the features or implementations described herein are necessary. The various features and implementations described herein are not meant to be taken in their limiting sense but are meant in their illustrative sense only. It will be apparent to those skilled in the art that certain features and sub-combinations are of utility and can be employed in applying the teachings of the present disclosure without recourse to the detailed construction and arrangement of the features herein described and illustrated.

[0200] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for video processing, comprising: dividing a timeline for video editing into a plurality of time segments according to material track segments on an edit track during a video editing process; performing a composite rendering on material track segments on the edit track within a target time segment of the plurality of time segments to generate video rendering data of the target time segment and cache the video rendering data; for each target time segment, generating a first identifier corresponding to the target time segment according to picture data of a material track segment corresponding to the target time segment; setting the first identifier to the cached video rendering data corresponding to the target time segment; for video rendering data of a target time segment, establishing a mapping relationship between the video rendering data and frame information of a material track segment in the target time segment; in response to receiving a video editing preview instruction, if a preview position is located within the target time segment, reading rendering data of the preview position from the cached video rendering data of the target time segment and displaying the rendering data. The response to receiving a video editing preview instruction, if a preview position is located within the target time segment, reading rendering data of the preview position from the cached video rendering data of the target time segment and displaying the rendering data, comprises: determining the first identifier corresponding to the target time segment; determining the video rendering data corresponding to the target time segment from the cache according to the first identifier of the target time segment; determining the rendering data corresponding to the preview position from the video rendering data corresponding to the target time segment according to the mapping relationship. 2.The method of claim 1, wherein: a start point of each material track segment is a start point of a time segment and a terminal point of a previous time segment of the time segment; a terminal point of each material track segment is a terminal point of a time segment and a start point of a next time segment of the time segment.

3. The method of claim 1, wherein, The mapping relationship between the rendering data of each time segment and frame information of a material track segment in the time segment is established by: determining frame information of the material track segment corresponding to the target time segment in a corresponding material; establishing a mapping relationship between the video rendering data of the target time segment and the frame information corresponding to the target time segment by using a second identifier of the video rendering data generated from the frame information.

4. The method of claim 3, wherein, The determination of the frame information of the material track segment corresponding to the target time segment in the corresponding material comprises: for a plurality of track material track segments within the target time segment, determining the frame information of the material track segment corresponding to the target time segment in the corresponding material according to frame information of a material track segment corresponding to the target time segment in the corresponding material.

5. The method of claim 1, wherein, The method further comprises: in response to receiving a re-edit operation performed on the material track segment, dividing the timeline according to information of the material track segment on the edit track; determining a new target time segment from the division result without overlapping with a historical target time segment, and generating video rendering data for the material track segment corresponding to the new target time segment and caching the video rendering data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: storing the video rendering data using a global storage path.

7. A video processing apparatus for performing the video processing method of any one of claims 1-6, comprising: a splitting unit configured to split a timeline for video editing into a plurality of time segments according to material track segments on an edit track during a video editing process; a rendering unit configured to perform a composite rendering on material track segments on the edit track within a target time segment of the plurality of time segments, generate video rendering data for the target time segment and cache the video rendering data; a display unit configured to, in response to receiving a video editing preview instruction, read rendering data for a preview position from the video rendering data cached for the target time segment and display the rendering data if the preview position is within the target time segment.

8. An electronic device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video processing method of any one of claims 1-6 is implemented.

10. A computer program product comprising a computer program, characterized in that, the computer program is executed by the processor to implement the video processing method of any one of claims 1-6.

Images