Video processing method and apparatus, and electronic device

Abstract

Embodiments of the present disclosure provide a video processing method and apparatus, and an electronic device. The method comprises: in a video editing process, on the basis of material track clips on an editing track, dividing a time axis for video editing into a plurality of time periods (S201); performing composite rendering on the material track clips within a target time period among the plurality of time periods on the editing track, generating video rendering data of the target time period, and caching the video rendering data (S202); and in response to reception of a video editing preview instruction, if a preview position is within the target time period, reading rendering data at the preview position from the video rendering data that has been cached within the target time period and displaying the rendering data (S203). The lag phenomenon caused by real-time rendering required when previewing the editing effect in the video editing process can be reduced, thereby improving the user experience in the video editing process.

Description

Video processing method, device and electronic equipment

[0001] This application claims priority to the Chinese invention patent application entitled “Video processing method, device and electronic device” and application number 202310804891.5 filed on June 30, 2023. The entire contents of that application are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a video processing method, apparatus, and electronic device. Background Art

[0003] Video editing usually involves adding special effects or scenes to recorded videos, or mixing video clips from different video sources to create a new video.

[0004] The video editing and editing effect previewing processes described above consume significant system resources. When displaying the edited results on the electronic device performing the video editing, screen freezes often occur. Users often preview the edited results frequently during the video editing process, and screen freezes may occur during these multiple previews, resulting in a poor video editing experience.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a video processing method, apparatus, and electronic device to overcome the problem of poor experience caused by freezes in previewing editing effects during video editing in related technologies.

[0007] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising: during a video editing process, dividing a timeline used for video editing into multiple time periods according to material track segments on an editing track; synthesizing and rendering the material track segments on the editing track that are within a target time period among the multiple time periods, generating and caching video rendering data for the target time segment; in response to receiving a video editing preview instruction, if a preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached for the target time period.

[0008] In a second aspect, an embodiment of the present disclosure provides an image processing device, comprising: a segmentation unit, for segmenting a timeline for video editing into multiple time periods according to material track segments on an editing track during a video editing process; a rendering unit, for synthesizing and rendering the material track segments on the editing track that are within a target time period among the multiple time periods, generating and caching video rendering data of the target time segment; and a display unit, for responding to receiving a video editing preview instruction, and if a preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period.

[0009] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing method as described in the first aspect and various possible methods involved in the first aspect.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0011] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned first aspect and various video processing methods that may be involved in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0013] FIG1 is a schematic diagram of a video editing interface provided by an embodiment of the present disclosure;

[0014] FIG2 is a flow chart of a video processing method according to an embodiment of the present disclosure;

[0015] FIG3 is a schematic diagram of time axis segmentation provided by an embodiment of the present disclosure;

[0016] FIG4 is a second flow chart of a video processing method provided by an embodiment of the present disclosure;

[0017] FIG5 is a schematic diagram of a scenario provided by the present disclosure;

[0018] FIG6 is a schematic diagram of a scenario provided by the present disclosure;

[0019] FIG7 is a structural block diagram of a video processing device provided by an embodiment of the present disclosure;

[0020] FIG8 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0022] Please refer to Figure 1, which shows a schematic diagram of a video editor. This diagram illustrates the process of mixing two video clips. The video editor mixes video clip 10 with video clip 20. Below the timeline 13 of the video editing interface may be tracks 11 and 12. Track 11 includes a clip consisting of multiple frames selected from video clip 10. Track 12 may display a clip consisting of multiple frames selected from video clip 20.

[0023] The user drags the material track segment at track 11 and the material track segment at video frame area 12 to select the frame at which the video material 10 and the video material 20 are to be mixed. After determining multiple mixed frames of the two, the effect can be previewed to browse the editing effect.

[0024] Figure 1 is just an example of video editing. In practice, multiple video materials can be mixed, or text, special effects graphics, etc. can be added to a video. During the video editing process, users usually preview the editing effects in order to view the editing effects. If the electronic device used by the user for video editing has low resource configuration, or the material used for video editing is large, when previewing the editing effects, each frame needs to be rendered and a preview screen is generated based on the rendering results. Since the rendering of each frame corresponding to the editing results requires more system resources, the preview editing results will be displayed in a stuck state, resulting in a poor user experience.

[0025] To address this issue, the solution provided by this disclosure divides the timeline into multiple time segments based on the material track segments on the editing track. Video rendering data is generated and cached for a target time segment within the multiple time segments. During preview, the video rendering data for the target time segment can be used to generate the preview image. Furthermore, the video rendering data processed in this manner can be reused, reducing the resources required to preview the video editing results and thus reducing lag.

[0026] Please refer to Figure 2, which is a flow chart of a video processing method provided by an embodiment of the present disclosure. The execution subject of the video processing method of the present disclosure may be a terminal device, and the method includes:

[0027] S201: During the video editing process, a timeline for video editing is divided into multiple time periods according to the material track segments on the editing track.

[0028] In this embodiment, various video editing applications can be run in the terminal device, in which video materials to be edited can be imported and editing operations can be performed on the video materials.

[0029] The video material to be edited can include video material obtained by various methods, including but not limited to: video material recorded using a video recording device, video material obtained from other electronic devices based on authorization, etc. The video material can also include one of the following: preset special effects, preset backgrounds, etc. The video material to be edited is described here only by way of example and is not an exhaustive list of all video materials. The above examples are not intended to limit the scope of protection of the disclosed solution.

[0030] In video editing applications, a timeline is often displayed within the editing interface. This refers to the horizontal list of thumbnails that appears below the video preview during editing and progresses as the video plays. The timeline allows you to control the content displayed at a specific point in the video. Furthermore, using the timeline and network technology, you can link multiple events in chronological order to form a relatively complete record system.

[0031] For a video material in the video editing material, after the video material is transcoded, multiple video frames corresponding to the video material are displayed and arranged according to the appearance time below the time axis. As shown in Figure 1, for video 1, multiple video frames of video material 10 can be displayed below the time axis 13.

[0032] During video editing, an editing track may include at least one material track clip.

[0033] Material track clips include text material track clips, graphic special effect material track clips, or video material track clips.

[0034] The above timeline can be split according to the material track segments on the editing track to obtain multiple time periods.

[0035] As an implementation, you can use the start and end points of a material track segment as split points, and then split the timeline based on these split points. For multiple material track segments, you can use the start and end points of each material track segment as split points. For material track segments on different tracks, you can use the start and end points of each material track segment as split points.

[0036] By dividing the time axis using the above division points, multiple time periods can be obtained.

[0037] In this way, the starting point of each material track segment is the starting point of a time period and the end point of the time period before the time period.

[0038] In the above time segmentation method, the end point of each material track segment is the end point of a time segment and the starting point of the next time segment after the time segment.

[0039] In each time period, there will not be more than one material track segment data in the same track in that time period.

[0040] Please refer to Figure 3, which shows a schematic diagram of a timeline segmentation. Different video clips can correspond to different tracks. For example, Video Clip 1 and Video Clip 2 can correspond to Track 1 and Track 2, respectively. Assume that the first frame of Track Clip 1 appears on Track 1 at 00:00 (t0) on the timeline (i.e., t0 is the start point of Track Clip 1). Starting at t1, the first frame of Track Clip 2 corresponding to Track Clip 2 appears in Video Clip 2 (i.e., t1 is the start point of Track Clip 2). At time t2, Track Clip 1 ends (t2 is the end point of Track Clip 1); and at time t3, Track Clip 2 ends (t3 is the end point of Track Clip 2). In this way, since the timeline can be segmented using t0, t1, t2, and t3 as the segmentation points, we get Time Segment 1 from t0 to t1; Time Segment 2 from t1 to t2; and Time Segment 3 from t2 to t3. t1 is the end point of Time Segment 1 and the start point of Time Segment 2. t2 is the starting point of time period 3 and also the ending point of time period 2.

[0041] The multiple time segments obtained through the above segmentation method have good continuity among the material track segments within each track within the time segment. Furthermore, each time segment obtained through the above segmentation method does not include multiple material track segments, thereby reducing the computational effort required to generate video rendering data for each time segment, thereby facilitating rapid generation of video rendering data for each time segment.

[0042] S202: Performing synthesis rendering on the material track segments within a target time segment among the multiple time segments on the editing track, generating and caching video rendering data of the target time segment.

[0043] After obtaining multiple time periods in step S101, a target time period can be determined among the multiple time periods. The target time period can be a time period corresponding to a larger number of material track segments among the multiple time periods, or a time period corresponding to a material track segment with more picture information changes.

[0044] For a target time period, the corresponding material track segments within the target time period can be determined, and then a synthesis rendering task can be generated. This synthesis rendering task can then be executed. Specifically, the corresponding material track segments within the target time period can be synthesized according to a preset synthesis rule, and then rendered to obtain video rendering data corresponding to the target time period.

[0045] For example, the video frames corresponding to the material track segments aligned on the timeline within the target time period are synthesized on a frame-by-frame basis and then rendered. Alternatively, the video frames and graphics frames in the material track segments aligned on the timeline are synthesized on a frame-by-frame basis and then rendered. Therefore, the video rendering data corresponding to the target time period may include synthesized rendering data corresponding to multiple frames within the target time period.

[0046] In practice, the rendering control management object provided by the video editing application can be called to perform the composite rendering task.

[0047] The composite rendering tasks corresponding to the multiple target time periods may be executed sequentially according to a serial task execution method, or may be executed in parallel according to a parallel task execution method.

[0048] For each target time period, the video rendering data corresponding to the target time period may be cached.

[0049] In some embodiments, the video processing method further includes: caching the video rendering data using a global storage path.

[0050] In these embodiments, the global storage path here may be a storage path other than the draft corresponding to the video material being edited.

[0051] After setting the global storage path, you can associate the video rendering data with the draft of the video material. In this way, when the draft name is changed, the video rendering data will not be affected by the draft name change.

[0052] S203: In response to receiving the video editing preview instruction, if the preview position is within the target time period, read and display the rendering data of the preview position from the video rendering data cached in the target time period.

[0053] When editing a video, the user often needs to view the editing effect. During the editing operation, the user can click on the preview control that can be used to preview the editing effect, thereby sending the preview instruction to the execution subject.

[0054] After receiving the preview instruction, the preview position indicated by the preview instruction can be determined. If the preview position indicated by the preview instruction is within the target time period, the rendering data corresponding to the preview position is read from the video rendering data cached in the target time period and displayed.

[0055] In this embodiment, during the video editing process, the timeline used for video editing is divided into multiple time periods based on the material track segments on the editing track; the material track segments on the editing track that are within a target time period among the multiple time periods are synthesized and rendered, and video rendering data for the target time segment is generated and cached; in response to receiving a video editing preview instruction, if the preview position is within the target time period, the rendering data for the preview position is read from the cached video rendering data for the target time period and displayed. Since the timeline is divided into multiple time periods based on the material track segments on the editing track, and then the material track segments within the target time period among the multiple time periods are synthesized and rendered, when previewing the video editing effect, a preview screen is generated according to the video rendering data corresponding to the target time period. This can reduce the stuttering phenomenon caused by the need for real-time rendering when previewing the editing effect during the video editing process, thereby improving the user experience during the video editing process.

[0056] Please refer to Figure 4, which is a second flow chart of a video processing method provided by an embodiment of the present disclosure. The execution subject of the video processing method of the present disclosure may be a terminal device, and the method includes:

[0057] S401: During the video editing process, a timeline for video editing is divided into multiple time periods according to the material track segments on the editing track.

[0058] S402: Performing synthesis rendering on the material track segments within a target time segment among the multiple time segments on the editing track, generating and caching video rendering data of the target time segment.

[0059] The specific implementation of steps S401 to S402 can refer to the relevant description of the embodiment shown in Figure 2 and will not be described here in detail.

[0060] S403: For each target time period, generate a first identifier corresponding to the target time period according to the picture data of the material track segment corresponding to the target time period; and set a first identifier for the cached video rendering data corresponding to the target time period.

[0061] The target time period may be a time period from the multiple time periods obtained in step S401. For example, the target time period may be a time period determined from the multiple time periods based on a user's selection operation. Alternatively, the target time period may be a time period determined based on the amount of screen information change within each time period.

[0062] The first identifier here can be a first identifier obtained using various digital signature algorithms.

[0063] Schematically, for time period 2 (target time period) in FIG3 , a digital signature algorithm can be applied to the image data in material track segment 1 and material track segment 2 corresponding to time period 2 to obtain a first identifier for time period 2. The first identifier obtained by the digital signature algorithm is generally a unique identifier.

[0064] Illustratively, the above digital signature algorithm may be a message digest algorithm (Message Digest Algorithm MD5).

[0065] The first identifier is unique and irreversible. When the video editing data input to the digital signature algorithm changes, such as adding or reducing video frames, or adding special effects or backgrounds, the first identifier obtained by the digital signature algorithm will change.

[0066] Through the above steps, for a target time period among the multiple time periods, the track material segments within the target time period can be synthesized and rendered to obtain video rendering data. The video rendering data for the target time period is cached and the above first identifier is set. Since the first identifier is unique, the video rendering data corresponding to the target time period can be accurately found in the cache by the above first identifier.

[0067] For example, for time period 2 in FIG. 2 , each frame of image data of material track segment 1 of video material 1 corresponding to time period 2 can be pre-combined with each frame of image data of material track segment 2 of video material 2 corresponding to time period 2 and then rendered to obtain video rendering data for time period 2. A digital signature algorithm can be used to determine a first identifier for time period 2 using multiple frames of user data in material track segment 1 and multiple frames of image data in material track segment 2 in time period 2, and a first identifier can be set for the cached video rendering data for time period 2.

[0068] Therefore, for the same time period, as long as the image data of the material track segments in each track within that time period remains unchanged, the first identifiers calculated using the same digital signature algorithm for the image data in the material track segments within that time period will be the same. Based on this, when previewing the editing results, the first identifiers can be used to accurately find the video rendering data corresponding to each target time period.

[0069] In addition, adding the first identifier to the identifier of the video rendering data of the target time period can avoid performing multiple synthesis renderings on the same material track segment in the same time period.

[0070] S404: In response to receiving the video editing preview instruction, if the preview position is within the target time period, determine a first identifier corresponding to the target time period.

[0071] S405: Determine rendering data corresponding to the preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0072] After receiving the video editing preview instruction, the time period within which the video editing preview instruction falls can be determined. For example, a first identifier for the time period can be generated based on the screen data of each material editing segment in the time period according to the algorithm used to generate the first identifier. The first identifier is then used to perform a match in the cache. If a match is successful, the time period is identified as the target time period. The video rendering data corresponding to the target time period is then extracted from the cache.

[0073] Then, the rendering data corresponding to the preview position is extracted from the video rendering data corresponding to the target time period.

[0074] Compared to the solution in the embodiment shown in FIG. 2 , this embodiment generates a first identifier associated with the material track segment frame data for the target time period and caches the video rendering data for the target time period according to the first identifier. Therefore, when a preview instruction is received, the video rendering data for the target time period can be accurately and quickly found from the cache based on the first identifier, and the rendering data corresponding to the preview position can be extracted. This improves the accuracy of the preview effect displayed by the preview instruction.

[0075] In some embodiments, the video processing method further includes: for the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and the above step S405 further includes the following steps:

[0076] First, according to the first identifier of the target time period, the video rendering data corresponding to the target time period is determined from the cache.

[0077] Secondly, the rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the above mapping relationship.

[0078] Here, establishing a mapping relationship between video rendering data and the frame information of the material track segment clearly defines which frames in the material a piece of video rendering data corresponds to. This facilitates locating the video rendering data corresponding to a target time period based on the first identifier when a preview instruction indicates rendering data corresponding to that target time period. Then, based on the frame information, the rendering data corresponding to the preview position indicated by the preview instruction can be extracted from the video rendering data, allowing for accurate and rapid display of the editing effect corresponding to the preview position.

[0079] In some embodiments, the above step S404 includes the following steps:

[0080] Determine the frame information of the material track segment corresponding to the target time period in the corresponding material; and establish a mapping relationship between the video rendering data and the frame information corresponding to the target time period using a second identifier of the video rendering data of the target time period generated from the frame information.

[0081] Illustratively, for the video rendering data in time period 1 in FIG3 , the frame information for time period 1 may be the frame information "1-5 video images" of the material track segment in video material 1. The second identifier may be, for example, "Video material 1 video rendering data corresponding frames (1-5.mp4)." By generating a second identifier including the frame information for the target time period and setting the second identifier as part of the identifier for the video rendering data for the target time period, a mapping relationship is established between the video rendering data for the target time period and the frame information corresponding to the target time period.

[0082] In these embodiments, a second identifier generated from frame information can be set for the video rendering data. The rendering data corresponding to the video frame in the target time period can then be quickly determined based on the second identifier. Therefore, the rendering data corresponding to the preview position can be quickly determined from the cache based on the second identifier, further improving the speed of previewing editing effects.

[0083] In some embodiments, the above-mentioned determining the frame information of the material track segment corresponding to the target time period in the corresponding material includes: for multiple-track material track segments within the target time period, based on the frame information of the material track segment that appears latest on the timeline in the corresponding material, determining the frame information of the material track segment corresponding to the target time period in the corresponding material.

[0084] In these embodiments, the editing track may include multiple tracks, and the material may include multiple materials.

[0085] The following is explained in conjunction with Figure 5, which is a schematic diagram of an application scenario provided by the present disclosure. During the editing process, the material track segment 3 corresponding to the target time period D on the timeline is a segment from the 100th frame to the 200th frame extracted from the video material 1. The material track segment 4 is a material track segment extracted from the video material 2. Among them, the material track segment of the video material 2 corresponding to the target time period D is a segment from the 75th frame to the 175th frame of the video material 2. The user performed a mixed editing operation on the material track segment 3 and the material track segment 4. On the editing track, the appearance time of the material track segment 4 is later than the appearance time of the material track segment 3. The first frame of the video material 4 can be used as the starting position for determining the frame information of the target time period D. This is because the video material 1 and the video material 2 are mixed at most starting from the first frame of the video material 2.

[0086] In addition, a first identifier E can be generated based on the image data of frames 100 to 200 of video material 1 and frames 75 to 175 of video material 2. A second identifier can be generated based on the frame information of frames 75 to 175 of video material 2. Video rendering data can be generated for material track segments 3 and 4 corresponding to the target time period D, and a first identifier and a second identifier can be set for the video rendering data. The second identifier is used to establish a mapping relationship between the frame information 75-175 and the video rendering data. The second identifier can be, for example, "75-175.mp4."

[0087] In these embodiments, when editing multi-track video materials, for a target time period, the frame mapping information between the video rendering data and the track material segments of the target time period is determined based on the frame information of the track material segment that appears latest on the timeline in the corresponding material. This allows the user to reuse the video rendering data generated by previous editing operations when performing subsequent editing operations on the same video material, thereby saving system resources.

[0088] In some embodiments, the video processing method further includes the following steps:

[0089] First, in response to receiving a re-editing operation performed on a material track segment, the timeline is segmented according to information of the material track segment on the editing track.

[0090] Secondly, a new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

[0091] In these embodiments, video editing usually requires multiple editing operations on the video material to finally generate a video editing result that satisfies the user.

[0092] After generating video rendering data based on the previous editing operation and performing a preview, the user can continue to perform editing operations on the video editing material.

[0093] After receiving the user's re-editing operation on the video editing material, the time axis is segmented again according to the segmentation process of step S401, and multiple segmentation results can be obtained.

[0094] For each segmentation result, it is determined whether the segmentation result overlaps with the historical target time period. If so, a new target time period and corresponding video rendering data are not generated. If part of the segmentation result overlaps with the historical target time period, a new target time period can be generated based on the non-overlapping portion. If no part of the segmentation result overlaps with the historical target time period, the segmentation result can be used as the new target time period.

[0095] For the new target time period, video rendering data is generated and cached. During caching, a first identifier and a second identifier are set for the video rendering data. The first identifier is generated based on the frame data of the new target time period, and the second identifier is generated based on the frame information of the material track segment corresponding to the new target time period in the corresponding material.

[0096] When a preview instruction for previewing the re-editing operation is received, for a target time period that coincides with the historical target time period, the video rendering data corresponding to the historical target time period may be reused for preview presentation.

[0097] In conjunction with Figure 5 and Figure 6, Figure 6 is a schematic diagram of an application scenario provided by the present disclosure. For Figure 5, the user performs a mixed editing operation on the 100th to 200th frames of video material 1 and the 75th to 175th frames of video material 2. In the subsequent editing operation, the video material 1 is moved to the left on the timeline, and the number of frames for mixing video material 1 and video material 2 is increased. As shown in Figure 6, after the user's editing operation again, the 75th to 200th frames of video material 1 are mixed with the 50th to 175th frames of video material 2. That is, video material 1 corresponds to material track segment 3', and video material 2 corresponds to material track segment 4, as shown in Figure 6.

[0098] For the editing operation shown in FIG6 , a new target time period F corresponding to the 75th to 99th frames of the video material 1 can be generated in the above manner. For the new target time period F, a first identifier P is generated based on the picture data of the 75th to 99th frames of the video material 1 and the 50th to 74th frames of the video material 2. A second identifier of the new target time period is generated based on the frame information of the track material segment 4 in the video material 2 corresponding to the new target time period F. Video rendering data of the new target time period is generated, and a mapping relationship between the video rendering data and the frame information corresponding to the new target time period is established using the second identifier. The above-mentioned video rendering data information is cached, and an identifier including the first identifier and the second identifier is used for the video rendering data.

[0099] For frames 100 to 200 of video material 1 and frames 75 to 175 of video material 2, the video rendering data corresponding to the original target time period D can be reused. This allows previewing the edit operation shown in FIG6 , directly using the video rendering data corresponding to target time period D generated by the historical edit operation shown in FIG5 . When the preview position is at target time period F, the newly generated video rendering data can also be used for previewing.

[0100] In these embodiments, for multiple editing operations, the video rendering data corresponding to the original historical target time period can be reused, thereby reducing the resources required for rendering for multiple editing operations. This allows the user to smoothly display the preview screen generated based on the video rendering data each time they preview.

[0101] The video processing method, device, and electronic device provided in this embodiment divide the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process; synthesize and render the material track segments on the editing track that are within the target time period among the multiple time periods, generate and cache the video rendering data of the target time segment; in response to receiving a video editing preview instruction, if the preview position is within the target time period, read the rendering data of the preview position from the cached video rendering data of the target time period and display it. Since the timeline is divided into multiple time periods according to the material track segments on the editing track, and then synthesize and render the material track segments within the target time period, when previewing the video editing effect, a preview video is generated according to the video rendering data corresponding to different target time periods. This can reduce the jamming phenomenon caused by the need for real-time rendering when previewing the editing effect during the video editing process, thereby improving the user experience during the video editing process.

[0102] Corresponding to the video processing method of the above embodiment, FIG7 is a structural block diagram of a video processing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Referring to FIG7, the device 70 includes: a segmentation unit 701, a rendering unit 702, and a display unit 703. Among them:

[0103] The segmentation unit 701 is used to segment the time axis used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process.

[0104] The rendering unit 702 is configured to synthesize and render the material track segments within a target time segment among the multiple time segments on the editing track, generate video rendering data of the target time segment, and cache the data.

[0105] The display unit 703 is configured to, in response to receiving the video editing preview instruction, read and display the rendering data of the preview position from the video rendering data cached in the target time period if the preview position is within the target time period.

[0106] In some embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period and the end point of the time period before the time period; the end point of each material track segment is the end point of a time period and the starting point of the time period after the time period.

[0107] In some embodiments of the present disclosure, the apparatus 70 further includes an identification configuration unit (not shown in the figure). The identification configuration unit is configured to: for each target time period, generate a first identification corresponding to the target time period based on the picture data of the material track segment corresponding to the target time period; and set the first identification for the cached video rendering data corresponding to the target time period.

[0108] In some embodiments of the present disclosure, the display unit 703 is further configured to: determine a first identifier corresponding to a target time period; and determine rendering data corresponding to a preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0109] In some embodiments of the present disclosure, the device 70 further includes a mapping unit (not shown in the figure), which is used to: establish a mapping relationship between the video rendering data of the target time period and the frame information of the material track segment in the target time period; and the display unit 703 is further used to: determine the video rendering data corresponding to the target time period from the cache according to the first identifier of the target time period; and determine the rendering data of the preview position from the video rendering data corresponding to the target time period according to the mapping relationship.

[0110] In some embodiments of the present disclosure, the mapping unit is further used to: determine the frame information of the material track segment corresponding to the target time period in the corresponding material; and use the second identifier of the video rendering data of the target time period generated by the frame information to establish a mapping relationship between the video rendering data and the frame information corresponding to the target time period.

[0111] In some embodiments of the present disclosure, the mapping unit is further used to: for the multi-track material track segments within the target time period, determine the frame information of the material track segments corresponding to the target time period in the corresponding material based on the frame information of the material track segments that appear latest on the timeline in the corresponding material.

[0112] In some embodiments of the present disclosure, the segmentation unit is further used to: in response to receiving a re-editing operation performed on the material track segment, segment the timeline according to the information of the material track segment on the editing track; the rendering unit is further used to: determine a new target time period that does not overlap with the historical target time period from the segmentation result, and generate and cache video rendering data for the material track segment corresponding to the new target time period.

[0113] In some embodiments of the present disclosure, the apparatus 70 further includes a storage unit (not shown in the figure), which is configured to store the video rendering data using a global storage path.

[0114] In order to implement the above embodiment, the embodiment of the present disclosure further provides an electronic device.

[0115] Referring to Figure 8 , a schematic diagram of the structure of an electronic device 800 suitable for implementing embodiments of the present disclosure is shown. The electronic device 800 may be various terminal devices or servers. The terminal device may include an extended reality device. For example, the extended reality device may include various wearable devices, such as head-mounted extended reality devices. The electronic device shown in Figure 8 is merely an example and should not limit the functionality or scope of use of the embodiments of the present disclosure.

[0116] As shown in Figure 8, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage device 808 to the random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0117] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although FIG8 shows an electronic device 800 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0118] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0119] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0120] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0121] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0122] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. The name of a unit does not, in some cases, limit the unit itself. For example, a segmentation unit may be described as "a unit that, during the video editing process, segments the timeline used for video editing into multiple time periods based on the material track segments on the editing track."

[0125] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0126] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] In a first aspect, according to one or more embodiments of the present disclosure, a video processing method is provided, comprising: during a video editing process, dividing a timeline used for video editing into multiple time periods according to material track segments on an editing track; synthesizing and rendering the material track segments within a target time period among the multiple time periods on the editing track, generating and caching video rendering data of the target time segment; in response to receiving a video editing preview instruction, if a preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period.

[0128] According to one or more embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period and the end point of the time period before the time period; the end point of each material track segment is the end point of a time period and the starting point of the time period after the time period.

[0129] According to one or more embodiments of the present disclosure, the method further includes: for each target time period, generating a first identifier corresponding to the target time period based on the picture data of the material track segment corresponding to the target time period; and setting a first identifier for the cached video rendering data corresponding to the target time period.

[0130] According to one or more embodiments of the present disclosure, in response to receiving a video editing preview instruction, if the preview position is within a target time period, the rendering data of the preview position is read from the video rendering data cached in the target time period and displayed, including: determining a first identifier corresponding to the target time period; and determining the rendering data corresponding to the preset preview position from the video rendering data corresponding to the first identifier read from the cache.

[0131] According to one or more embodiments of the present disclosure, the method further includes: establishing, for video rendering data of a target time period, a mapping relationship between the video rendering data and frame information of a material track segment in the target time period; and determining rendering data corresponding to a preview position from the video rendering data corresponding to a first identifier read from a cache, including: determining, from the cache, the video rendering data corresponding to the target time period according to the first identifier of the target time period; and determining, according to the mapping relationship, the rendering data of the preview position from the video rendering data corresponding to the target time period.

[0132] According to one or more embodiments of the present disclosure, for rendering data of each time period, a mapping relationship is established between the rendering data and the frame information of the material track segment in the time period, including: determining the frame information of the material track segment corresponding to the target time period in the corresponding material; and using a second identifier of the video rendering data of the target time period generated from the frame information to establish a mapping relationship between the video rendering data and the frame information corresponding to the target time period.

[0133] According to one or more embodiments of the present disclosure, determining frame information of a material track segment corresponding to the target time period in the corresponding material includes: for multiple material track segments within the target time period, determining frame information of the material track segment corresponding to the target time period in the corresponding material based on the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0134] According to one or more embodiments of the present disclosure, the method further includes: in response to receiving a re-editing operation performed on a material track segment, segmenting the timeline according to information of the material track segment on the editing track; determining a new target time period that does not overlap with the historical target time period from the segmentation result, and generating and caching video rendering data for the material track segment corresponding to the new target time period.

[0135] According to one or more embodiments of the present disclosure, the method further includes: storing the video rendering data using a global storage path.

[0136] In a second aspect, according to one or more embodiments of the present disclosure, a video processing device is provided, including: a segmentation unit for segmenting a timeline for video editing into multiple time periods according to the material track segments on the editing track during the video editing process; a rendering unit for synthesizing and rendering the material track segments within a target time period among the multiple time periods on the editing track, generating and caching video rendering data of the target time segment; and a display unit for responding to receiving a video editing preview instruction, and if the preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period.

[0137] According to one or more embodiments of the present disclosure, the starting point of each material track segment is the starting point of a time period and the end point of the time period before the time period; the end point of each material track segment is the end point of a time period and the starting point of the time period after the time period.

[0138] According to one or more embodiments of the present disclosure, the apparatus further includes an identification configuration unit configured to: for each target time period, generate a first identification corresponding to the target time period based on the image data of the material track segment corresponding to the target time period; and set the first identification for the cached video rendering data corresponding to the target time period.

[0139] According to one or more embodiments of the present disclosure, the display unit is further configured to: determine a first identifier corresponding to a target time period; and determine rendering data corresponding to a preset preview position from video rendering data corresponding to the first identifier read from a cache.

[0140] According to one or more embodiments of the present disclosure, the video processing device further includes a mapping unit, which is configured to: establish, for the video rendering data of a target time period, a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; the display unit is further configured to: determine, from a cache, the video rendering data corresponding to the target time period based on a first identifier of the target time period; and determine, based on the mapping relationship, rendering data for a preview position from the video rendering data corresponding to the target time period.

[0141] According to one or more embodiments of the present disclosure, the mapping unit is further used to: determine the frame information of the material track segment corresponding to the target time period in the corresponding material; and use the second identifier of the video rendering data of the target time period generated by the frame information to establish a mapping relationship between the video rendering data and the frame information corresponding to the target time period.

[0142] According to one or more embodiments of the present disclosure, the mapping unit is further used to: for the multi-track material track segments within the target time period, determine the frame information of the material track segment corresponding to the target time period in the corresponding material based on the frame information of the material track segment that appears latest on the timeline in the corresponding material.

[0143] According to one or more embodiments of the present disclosure, the segmentation unit is further used to: in response to receiving a re-editing operation performed on the material track segment, segment the timeline according to the information of the material track segment on the editing track; the rendering unit is further used to: determine a new target time period that does not overlap with the historical target time period from the segmentation result, and generate and cache video rendering data for the material track segment corresponding to the new target time period.

[0144] The video processing device further includes a storage unit, which is configured to store the video rendering data using a global storage path.

[0145] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory; the memory stores computer-executable instructions; and at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing methods as described in the first aspect above and various possible methods involved in the first aspect.

[0146] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video processing method as described in the first aspect and various possible methods involved in the first aspect are implemented.

[0147] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the above first aspect and various video processing methods that may be involved in the first aspect.

[0148] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0149] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0150] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, comprising: During the video editing process, the timeline used for video editing is divided into multiple time periods according to the material track clips on the editing track; Performing synthesis rendering on the material track segments on the editing track that are located in a target time segment among the multiple time segments, generating and caching video rendering data of the target time segment; In response to receiving the video editing preview instruction, if the preview position is within the target time period, the rendering data of the preview position is read from the video rendering data cached in the target time period and displayed.

2. The method according to claim 1, wherein: The starting point of each of the material track segments is the starting point of a time period and the end point of the time period before the time period; The end point of each material track segment is the end point of a time period and the starting point of the next time period after the time period.

3. The method according to claim 1, wherein: The method further comprises: For each target time period, a first identifier corresponding to the target time period is generated according to the picture data of the material track segment corresponding to the target time period; and the first identifier is set for the cached video rendering data corresponding to the target time period.

4. The method according to claim 3, wherein: In response to receiving the video editing preview instruction, if the preview position is within the target time period, reading and displaying the rendering data of the preview position from the video rendering data cached in the target time period, comprises: Determine a first identifier corresponding to the target time period; The rendering data corresponding to the preview position is determined from the video rendering data corresponding to the first identifier read from the cache.

5. The method according to claim 4, wherein: The method further comprises: For the video rendering data of the target time period, establishing a mapping relationship between the video rendering data and the frame information of the material track segment in the target time period; and The determining, from the video rendering data corresponding to the first identifier read from the cache, the rendering data corresponding to the preview position includes: According to the first identifier of the target time period, determine from the cache the time period corresponding to the target time period Video rendering data; The rendering data of the preview position is determined from the video rendering data corresponding to the target time period according to the mapping relationship.

6. The method according to claim 5, wherein: The step of establishing a mapping relationship between the rendering data of each time period and the frame information of the material track segment in the time period includes: Determine the frame information of the material track segment corresponding to the target time period in the corresponding material; A mapping relationship between the video rendering data and the frame information corresponding to the target time period is established by using the second identifier of the video rendering data of the target time period generated by the frame information.

7. The method according to claim 6, wherein: The determining of frame information of the material track segment corresponding to the target time period in the corresponding material includes: For the multi-track material track segments within the target time period, the frame information of the material track segments corresponding to the target time period in the corresponding material is determined according to the frame information of the material track segment that appears latest on the timeline in the corresponding material.

8. The method according to claim 1, wherein: The method further comprises: In response to receiving a re-editing operation performed on the material track segment, segmenting the timeline according to information of the material track segment on the editing track; A new target time period that does not overlap with the historical target time period is determined from the segmentation results, and video rendering data is generated and cached for the material track segment corresponding to the new target time period.

9. The method according to any one of claims 1 to 8, wherein: The method further comprises: The video rendering data is stored using a global storage path.

10. A video processing device, comprising: A split unit is used to split the timeline used for video editing into multiple time periods according to the material track segments on the editing track during the video editing process; A rendering unit, configured to perform synthesis rendering on the material track segments on the editing track within a target time segment among the multiple time segments, generate video rendering data of the target time segment, and cache the data; The display unit is used for, in response to receiving a video editing preview instruction, reading and displaying rendering data of the preview position from the video rendering data cached in the target time period if the preview position is within the target time period.

11. An electronic device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the video processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the video processing method according to any one of claims 1 to 9 is implemented.

Images