Video processing method and device

Abstract

The embodiment of the invention provides a video processing method and related equipment / products, and relates to the technical field of data processing. The video processing method comprises the steps that a video editing interface is displayed, the video editing interface is used for displaying a timeline and a plurality of materials distributed on the timeline, and each material has a corresponding display area on the timeline; in response to an interaction operation for the timeline, determining the current display time and the density and / or variation amplitude of the material corresponding to the current display time; wherein the current display time is located in the display area of the corresponding material; determining the scaling of the timeline according to the density and / or variation amplitude of the corresponding material; and adjusting the timeline according to the scaling so as to adjust the display area of the corresponding material. According to the technical scheme of the embodiment of the invention, the timeline can be dynamically zoomed according to different editing requirements automatically, and optimized display of the materials is realized.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of data processing, and in particular, to a video processing method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] During video editing, the timeline, as a core component, is used to display the arrangement order of multiple materials and visualize the duration (i.e., the display area) of each material. However, existing video editing software still requires manual adjustment of the zoom ratio of the timeline to view editing details at different levels, which is cumbersome to operate, cannot be automatically adjusted, and provides a poor editing experience.

[0003] It should be noted that the above content is not necessarily prior art and does not limit the patent protection scope of the present application. Summary of the Invention

[0004] Embodiments of the present application provide a video processing method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the above technical problems.

[0005] One aspect of embodiments of the present application provides a video processing method, the method comprising: Displaying a video editing interface for displaying a timeline and a plurality of materials distributed on the timeline, each material having a corresponding display area on the timeline; In response to an interaction operation on the timeline, determining a current display time and the density and / or variation amplitude of the material corresponding to the current display time; wherein the current display time is within the display area of the corresponding material; Determining a zoom ratio of the timeline according to the density and / or variation amplitude of the corresponding material; Adjusting the timeline according to the zoom ratio to adjust the display area of the corresponding material.

[0006] Optionally, the material includes elements; determining the density of the material corresponding to the current display time includes: Determining the number of elements corresponding to the current display time; wherein the current display time is within the display area of the corresponding element; Determining the density according to the number of the corresponding elements.

[0007] Optionally, determining the number of elements corresponding to the current display time includes: Obtain the quantity of the corresponding elements from a pre-constructed first array according to the current display time; wherein, the timeline includes multiple first time points, the first array includes the quantities of the elements corresponding to the multiple first time points respectively, and the current display time matches one of the first time points. Wherein, the first array is constructed through the following operations: Determine multiple first time points on the timeline according to a first preset time interval; For each first time point, determine the corresponding element, and associate and write the quantity of the corresponding element with the first time point into the first array; wherein, the first time point is within the display area of the corresponding element.

[0008] Optionally, the video processing method further includes: In response to a first editing operation, obtain the edited element; wherein, the first editing operation includes adding and / or deleting elements; Update the first array based on the display area of the edited element and the multiple first time points.

[0009] Optionally, the material includes a video, and the video includes multiple video frames; determining the change amplitude of the material corresponding to the current display time includes: Determine the image error between the current frame and the adjacent frame according to the current display time; wherein, there is a second preset time interval between the adjacent frame and the current frame and the adjacent frame is earlier than the current frame; Determine the change amplitude according to the image error between the current frame and the adjacent frame.

[0010] Optionally, determining the image error between the current frame and the adjacent frame according to the current display time includes: Obtain the image error between the current frame and the adjacent frame from a pre-constructed second array according to the current display time; wherein, the timeline includes multiple second time points, the second array includes the image errors between each second time point and the frame corresponding to the previous second time point respectively, and the current display time matches one of the second time points; Wherein, the second array is constructed through the following operations: Determine multiple second time points on the timeline according to a second preset time interval; For each second time point, determine the image error between the corresponding frame and the frame corresponding to the previous second time point, and associate and write the image error with the second time point into the second array.

[0011] Optionally, the video processing method further includes: In response to a second editing operation, obtain the edited video; wherein, the second editing operation includes adding, deleting, and / or replacing videos. Based on multiple video frames of the edited video and the multiple second time points, update the second array.

[0012] Optionally, the material includes elements and videos, the video includes multiple video frames, the density of the corresponding material is the number of the corresponding elements, and the change range of the corresponding material is the change range of the current frame. Determine the scaling ratio of the timeline according to the density and / or change range of the corresponding material, including: Compare the number of the corresponding elements with a first preset threshold. When the number of the corresponding elements is greater than the first preset threshold, determine the magnification ratio of the timeline. When the number of the corresponding elements is less than the first preset threshold, determine the average change range according to the multiple video frames; compare the ratio of the change range of the current frame to the average change range with a second preset threshold; when the ratio is greater than the second preset threshold, determine the magnification ratio of the timeline.

[0013] Optionally, adjust the timeline according to the scaling ratio to adjust the display area of the corresponding material, including: Enlarge the timeline according to the magnification ratio to enlarge the display area of the corresponding material. In response to an interaction operation on the enlarged timeline, obtain the latest current display time and the density and / or change range of the latest corresponding material. When the density is less than the first preset threshold and the ratio of the change range to the average change range is less than the second preset threshold, determine the reduction ratio of the timeline. Reduce the timeline according to the reduction ratio to reduce the display area of the latest corresponding material.

[0014] Another aspect of the embodiments of the present application provides a video processing device, and the device includes: A display module, configured to display a video editing interface, where the video editing interface is used to display a timeline and multiple materials distributed on the timeline, and each material has a corresponding display area on the timeline. A first determination module, configured to determine a current display time, and the density and / or variation amplitude of the material corresponding to the current display time in response to an interaction operation on the timeline; wherein, the current display time is within the display area of the corresponding material; A second determination module, configured to determine the zoom ratio of the timeline according to the density and / or variation amplitude of the corresponding material; An adjustment module, configured to adjust the timeline according to the zoom ratio to adjust the display area of the corresponding material.

[0015] Another aspect of the embodiments of the present application provides a computer device, including: At least one processor; and A memory communicatively connected to the at least one processor; Wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0016] Another aspect of the embodiments of the present application provides a computer-readable storage medium, in which computer instructions are stored, and when the computer instructions are executed by a processor, the method as described above is implemented.

[0017] Another aspect of the embodiments of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method as described above is implemented.

[0018] The embodiments of the present application adopting the above technical solutions may include the following advantages: The timeline and a plurality of materials distributed on the timeline are displayed through a video editing interface, and each material has a corresponding display area on the timeline. In response to an interaction operation on the timeline, the current display time and the density and / or variation amplitude of the corresponding material are determined. Wherein, the current display time is within the display area of the corresponding material. Based on the density and / or variation amplitude of the corresponding material, the zoom ratio of the timeline is determined. The timeline is adjusted according to the zoom ratio to adjust the display area of the corresponding material. It can be seen that in the embodiments of the present application, the timeline is dynamically zoomed according to the density and / or variation amplitude of the material corresponding to the selected position (current display time) of the timeline, and the display area of the corresponding material is also zoomed accordingly. There is no need for the user to manually adjust, and it automatically adapts to different editing requirements, realizing the optimized display of materials and improving the flexibility and convenience of video editing. Description of the Drawings

[0019] The accompanying drawings exemplarily illustrate embodiments and form a part of the specification, and are used together with the written description of the specification to explain the exemplary embodiments of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. In all the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 Schematically shows a flowchart of a video processing method according to Embodiment 1 of the present application; Figure 2 Schematically shows the magnified state of the timeline of a video editing interface according to Embodiment 1 of the present application; Figure 3 Schematically shows the normal state of the timeline of a video editing interface according to Embodiment 1 of the present application; Figure 4 Schematically shows Figure 1 The flowchart of the sub-steps of sub-step S102; Figure 5 Schematically shows an additional flowchart of a video processing method according to Embodiment 1 of the present application; Figure 6 Schematically shows Figure 1 The flowchart of the sub-steps of sub-step S102; Figure 7 Schematically shows an additional flowchart of a video processing method according to Embodiment 1 of the present application; Figure 8 Schematically shows a block diagram of a video processing apparatus according to Embodiment 2 of the present application; and Figure 9 Schematically shows a schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of the present application. Detailed implementation manners

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0022] It should be noted that in the embodiments of the present application, the descriptions involving "first", "second", etc. are only for descriptive purposes, and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. Additionally, the technical solutions between various embodiments may be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0023] In the description of the present application, it should be understood that the numerical labels before the steps do not identify the sequence of execution of the steps, but are only used to facilitate the description of the present application and distinguish each step, and thus should not be construed as a limitation to the present application.

[0024] First, the following provides the explanations of the terms involved in the present application: MSE (Mean Squared Error): Mean Squared Error.

[0025] RMSE (Root Mean Squared Error): Root Mean Squared Error.

[0026] PSNR (Peak Signal-to-Noise Ratio): Peak Signal-to-Noise Ratio.

[0027] Secondly, to facilitate the understanding of the technical solutions provided in the embodiments of the present application by those skilled in the art, the related technologies are described below: During the video editing process, the timeline, as a core component, is used to display the arrangement order of multiple materials and visualize the duration of each material (i.e., the display area). However, the applicant has learned that related video editing software still needs to manually adjust the zoom ratio of the timeline to view different levels of editing details, which is cumbersome to operate, cannot achieve automatic adjustment, and the editing experience is poor. Some video editing software can only perform simple zooming according to preset rules. For example, when the editor is initialized, the appropriate zoom value is automatically calculated based on the length of the timeline, but it cannot be adjusted after the setting is completed, which affects the editing efficiency.

[0028] To this end, the embodiments of the present application provide a technical solution for video processing. In this technical solution: (1) It is possible to dynamically adjust the zoom ratio of the timeline according to the number of elements (density) at the selected position (current display time) on the timeline and the change amplitude of the video content, so as to achieve an optimized display of the video editing content, that is, to provide a more intuitive editing interface, enabling editors to locate and process video details faster, and improving the editing efficiency and editing experience. (2) It can automatically adapt to the editing requirements of different video scenarios without manual adjustment by the editor, improving the flexibility and convenience of the editing process. See the following for details.

[0029] The technical solution of the present application will be introduced below through multiple embodiments. It should be noted that these embodiments can be implemented in many different forms and should not be construed as being limited only to the embodiments described herein.

[0030] Embodiment 1 Figure 1 The flowchart of the video processing method according to Embodiment 1 of the present application is schematically shown.

[0031] As Figure 1 shown, the video processing method may include steps S100 to S106, where: Step S100, display a video editing interface, where the video editing interface is used to display a timeline and a plurality of materials distributed on the timeline, and each material has a corresponding display area on the timeline.

[0032] Step S102, in response to an interaction operation on the timeline, determine the current display time and the density and / or change amplitude of the material corresponding to the current display time; where the current display time is located within the display area of the corresponding material.

[0033] Step S104, determine the zoom ratio of the timeline according to the density and / or change amplitude of the corresponding material.

[0034] Step S106, adjust the timeline according to the zoom ratio to adjust the display area of the corresponding material.

[0035] The video processing method provided in this embodiment displays a timeline and multiple materials distributed on the timeline through a video editing interface, and each material has a corresponding display area on the timeline. In response to an interaction operation on the timeline, the current display time and the density and / or change amplitude of the corresponding material are determined. Among them, the current display time is located within the display area of the corresponding material. Based on the density and / or change amplitude of the corresponding material, the scaling ratio of the timeline is determined. The timeline is adjusted according to the scaling ratio to adjust the display area of the corresponding material. It can be seen that the embodiment of the present application dynamically scales the timeline according to the density and / or change amplitude of the material corresponding to the selected position (current display time) of the timeline, and the display area of the corresponding material is also scaled accordingly. There is no need for the user to manually adjust, and it automatically adapts to different editing requirements, realizing an optimized display of the material and improving the flexibility and convenience of video editing.

[0036] The following combines Figure 1 , and elaborates on each step in steps S100 to S106 and other optional steps in detail.

[0037] Step S100 , display a video editing interface, and the video editing interface is used to display a timeline and multiple materials distributed on the timeline, and each material has a corresponding display area on the timeline.

[0038] In video editing, a material can be an object used to construct a finished film. For example, materials can include videos and elements. Among them, a video can include one or more video clips. Elements can include stickers, text, special effects, music, recordings, transitions, etc., which can be used to enrich video content. As Figure 2 , Figure 3 shown, the timeline can be the core component of the video editing interface. Its essence can be a linear area, which can be used to display the arrangement order of multiple materials, and can also clearly and visually display the duration (i.e., the display area) of each material. Among them, the display area can have corresponding in-points and out-points. The in-point can be the time when an element appears on the timeline (start display), and the out-point can be the time when an element disappears on the timeline (end display). Through the timeline, it is possible to conveniently integrate materials such as tracks (such as video tracks) and common tools (such as stickers, text, special effects) together to form an intuitive and efficient timeline panel, thereby improving video editing efficiency.

[0039] Step S102 , in response to an interaction operation on the timeline, determine the current display time and the density and / or change amplitude of the material corresponding to the current display time; among them, the current display time is located within the display area of the corresponding material.

[0040] As Figure 2 andFigure 3 As shown, the timeline can also provide a visual tool, the "playhead" (the vertical line in the figure). The playhead can be dragged to different positions on the timeline or move along the timeline when playing the video to help locate the current display time. Exemplarily, the creator can drag the playhead and move it to any position on the timeline to determine the current display time. According to the current display time and the display areas corresponding to multiple materials, the density and / or the change range of the materials corresponding to the current display time can be determined. Multiple exemplary solutions are provided below.

[0041] In an alternative embodiment, as Figure 4 shown, step S102 may include: Step S400, determining the number of elements corresponding to the current display time; wherein, the current display time is within the display area of the corresponding elements.

[0042] Step S402, determining the density according to the number of the corresponding elements.

[0043] Exemplarily, the corresponding elements can be determined according to the current display time. Specifically, for each element, it can be compared whether the current display time is within the display area of the element (i.e., between the in point and the out point). If it is within the display area of the element, the element can be determined as the element corresponding to the current display time; if it is not within the display area of the element, it can be regarded as an element irrelevant to the current display time. Count the number of elements corresponding to the current display time, and the number can be used to characterize the density.

[0044] In this embodiment, by counting the number of elements, the density can be effectively measured, which is subsequently used to calculate the scaling ratio of the timeline, helping the creator to accurately control the display area of the materials, thereby optimizing the editing effect and the editing experience.

[0045] The above embodiment exemplarily introduces determining the number of elements through real-time calculation. To further improve the editing efficiency, the number of elements can also be determined based on a fast query of preprocessing. Exemplary solutions are provided below.

[0046] In an alternative embodiment, step S400 may include: obtaining the quantity of the corresponding element from a pre-constructed first array according to the current display time; wherein the timeline includes a plurality of first time points, the first array includes the quantities of the elements corresponding to the respective plurality of first time points, and the current display time matches one of the first time points. The first array is constructed by the following operations: determining a plurality of first time points on the timeline according to a first preset time interval; for each first time point, determining the corresponding element, and associating and writing the quantity of the corresponding element with the first time point into the first array; wherein the first time point is within the display area of the corresponding element.

[0047] Exemplarily, a first time point can be determined every first preset time interval (such as 0.2 s) to obtain a plurality of first time points. For each first time point, all elements on the timeline can be traversed once. It is judged whether the first time point is between the in-point and out-point of the element (within the display area). If so, the element is used as the element corresponding to the first time point, and the corresponding quantity count is incremented by 1. Putting the total quantity of the elements corresponding to the first time point into the first array, an array arranged in chronological order can be obtained, and this array records the quantities of the elements corresponding to each first time point. Obtaining the ratio of the current display time t to the first preset time interval and taking the integer can obtain an index value. According to the index value, the first time point can be quickly matched from the first array, and the quantity of the corresponding element can be read. For example: the current display time t can be 0.67 s (670 ms), the first preset time interval can be 0.2 s (200 ms), and the index value obtained after taking the integer of the ratio can be 3, that is, the third first time point in the first array matches the current display time, and the quantity of the element corresponding to this first time point is the quantity of the element corresponding to the current display time.

[0048] In this embodiment, by storing the element quantities arranged in chronological order in the first array, the quantity of the element corresponding to the current display time can be efficiently obtained during the editing process, reducing the real-time calculation overhead and significantly improving the editing efficiency.

[0049] In an alternative embodiment, as Figure 5 described, the video processing method may further include: Step S500, in response to a first editing operation, obtaining the edited element; wherein the first editing operation includes adding and / or deleting elements.

[0050] Step S502, updating the first array based on the display area of the edited element and the plurality of first time points.

[0051] Exemplarily, in the case where an editor performs a first editing operation (such as adding an element, deleting an element, or adjusting the position of an element on the timeline), the edited multiple elements are determined. Similarly, for each first time point, all the edited elements on the timeline can be traversed once. It is determined whether the first time point is between the in-point and out-point of the edited element (within the display area) to determine the number of elements corresponding to the first time point, thereby updating the first array.

[0052] In this embodiment, dynamically updating the element quantity array after the editing operation can ensure the accuracy of the number of elements in the first array, thereby improving the flexibility and precision of video editing.

[0053] The above-mentioned multiple embodiments introduce that the density is determined by the number of elements. Next, the method for determining the change range will be introduced in combination with multiple embodiments.

[0054] In an alternative embodiment, as Figure 6 described, step S102 may include: Step S600, determining the image error between the current frame and the adjacent frame according to the current display time; wherein, there is a second preset time interval between the adjacent frame and the current frame and the adjacent frame is earlier than the current frame.

[0055] Step S602, determining the change range according to the image error between the current frame and the adjacent frame.

[0056] Exemplarily, the current frame and adjacent frames can be determined according to the current display time. The current frame corresponds to the current display time, and the adjacent frame is earlier than the current frame. There may be a second preset time interval (such as 0.2 s) between the adjacent frame and the current frame. It should be noted that the first preset time interval and the second preset time interval may be the same or different, and can be specifically set and adjusted according to actual needs. Calculate the image error between the current frame and the adjacent frame. The image error can be used to characterize the change amplitude of the video content. There are various methods for calculating the image error. For example, the average value of the squares of the differences between the pixels at the same positions of two images can be calculated, that is, the mean square error MSE. The smaller the MSE, the more similar the two images are, and the smaller the change amplitude of the video content. The larger the MSE, the greater the difference between the two images, and the larger the change amplitude of the video content. The square root of the mean square error MSE can also be taken to obtain the root mean square error RMSE to obtain a measurement value consistent with the original pixel unit, so that the image error is more intuitive and easier to understand and interpret. Further, the difference in image quality can also be measured by comparing the dynamic ranges of two images (the brightness difference between the brightest and darkest parts of the images) and the mean square error. The higher the value of this index, the smaller the difference between the two images, and the smaller the change amplitude of the video content. The lower the value of this index, the greater the difference between the two images, and the larger the change amplitude of the video content.

[0057] In this embodiment, by calculating the image error between the current frame and the adjacent frame, the change amplitude of the video content can be accurately quantified, and then used to calculate the scaling ratio of the timeline to help the creator accurately control the display area of the material, further optimizing the editing effect and editing experience.

[0058] The above embodiments exemplarily introduce determining the change amplitude through real-time calculation. To further improve the editing efficiency, the change amplitude can also be determined based on a fast query of preprocessing. The following provides an exemplary solution.

[0059] In an alternative embodiment, step S602 may include: obtaining the image error between the current frame and the adjacent frame from a pre-constructed second array according to the current display time; wherein, the timeline includes multiple second time points, and the second array includes the image errors between the frames corresponding to each second time point and the frame corresponding to the previous second time point, and the current display time matches one of the second time points; wherein, the second array is constructed by the following operations: determining multiple second time points on the timeline according to the second preset time interval; for each second time point, determining the image error between the corresponding frame and the frame corresponding to the previous second time point, and associating and writing the image error with the second time point into the second array.

[0060] Exemplarily, a second time point can be determined at every second preset time interval (such as 0.2 s), and multiple second time points are obtained. For each second time point, the corresponding frame is intercepted. The image error is calculated for the corresponding frames of two adjacent time points, and the image error is associated with the latter of the two adjacent time points and placed in the second array, so as to obtain an array arranged in chronological order, and this array records the image error corresponding to each second time point. The ratio of the current display time t to the first preset time interval is obtained, and the integer part is taken to obtain the index value. According to the index value, the second time point can be quickly matched from the second array, and the image error can be read. For example: the current display time t can be 1.21 s, the first preset time interval can be 0.2 s, and the index value obtained after taking the integer part of the ratio can be 6, that is, the sixth second time point in the second array is matched with the current display time, and the image error corresponding to this second time point is the change range of the video content corresponding to the current display time.

[0061] In this embodiment, by storing the image error array arranged in chronological order through the second array, the change range of the video content can be quantified quickly and accurately, which can be efficiently obtained during the editing process, reduce the real-time calculation overhead, and significantly improve the editing efficiency.

[0062] In an alternative embodiment, as Figure 7 shown, the video processing method includes: Step S700, in response to a second editing operation, obtain the edited video; wherein, the second editing operation includes adding, deleting, and / or replacing video.

[0063] Step S702, update the second array based on multiple video frames of the edited video and the multiple second time points.

[0064] Exemplarily, in the case where an editor performs a second editing operation (such as adding, deleting, and / or replacing video / video segments), the edited video is determined. Similarly, the image error between the latest corresponding video frames of two adjacent time points is calculated, thereby updating the second array.

[0065] In this embodiment, by dynamically updating the image error array after the editing operation, the impact of the editing on the change of the video content can be reflected in real time, ensuring the accuracy of the image error array, and further improving the flexibility and precision of video editing.

[0066] Step S104 , determine the scaling ratio of the timeline according to the density and / or change range of the corresponding material.

[0067] Exemplarily, in the case where the density of the corresponding material is relatively high or the change range is relatively large, the editor needs to frequently zoom in on the timeline to view the details of the material. When the density of the corresponding material is relatively low or the change range is small, the editor may need to zoom out the timeline to view a wider time range. Therefore, the zoom ratio of the timeline can be determined according to the density and / or change range of the corresponding material. An exemplary solution is provided below.

[0068] In an alternative embodiment, step S104 may include: Step S800, comparing the number of the corresponding elements with a first preset threshold.

[0069] Step S802, when the number of the corresponding elements is greater than the first preset threshold, determining the zoom ratio of the timeline.

[0070] Step S804, when the number of the corresponding elements is less than the first preset threshold, determining the average change range according to the multiple video frames; comparing the ratio of the change range of the current frame to the average change range with a second preset threshold; when the ratio is greater than the second preset threshold, determining the zoom ratio of the timeline.

[0071] Exemplarily, the number n of the elements corresponding to the current display time can be read from the first array, and the number of the elements is compared with a first preset threshold (which can be set according to actual needs, such as 3). When it is greater than the first preset threshold (such as n≥3), the zoom ratio of the timeline can be determined (such as horizontally magnifying by 2 times). The zoom ratio is used to indicate that the timeline is magnified so that the elements can be clearer. When it is less than the first preset threshold (such as n<3), the image error w of the frame corresponding to the current display time is taken out from the second array. At the same time, the average value p of the second array is calculated. If w / p≥1.5 (the second preset threshold), it is considered that the video content changes greatly, and the zoom ratio of the timeline can be determined, horizontally magnifying by 2 times.

[0072] In this embodiment, by dynamically adjusting the zoom ratio of the timeline according to the number of elements and the change range of the video content, the display effect of the material can be optimized, and the editing accuracy and efficiency can be improved.

[0073] Step S106 , adjusting the timeline according to the zoom ratio to adjust the display area of the corresponding material.

[0074] Exemplarily, adjusting the timeline according to the zoom ratio to adjust the display area of the corresponding material, such as Figure 2As shown. In an optional embodiment, the creator can slide the playback head of the magnified timeline to determine the latest current display time and the density and / or change range of the corresponding latest material. In the case where the density is less than the first preset threshold and the ratio of the change range to the average change degree is less than the second preset threshold, the reduction ratio of the timeline is determined. The timeline is reduced according to the reduction ratio to reduce the display area of the corresponding latest material, as Figure 3 shown.

[0075] In this embodiment, the scaling ratio of time can be dynamically adjusted according to the number of elements and the change range of the video content. When there are more elements or the change range of the video content is large, the timeline can be automatically magnified to view details. When there are fewer elements or the change of the video picture is small, the timeline can be automatically reduced to restore normal display.

[0076] In some embodiments, an option for manually adjusting the scaling ratio can also be provided to meet the personalized needs of different creators; shortcut keys or menu options can also be set to facilitate quickly adjusting the scaling state of the timeline.

[0077] To make the present application easier to understand, an exemplary application is provided below.

[0078] S1: Starting from the starting point of the video editing timeline, a time point is determined every 0.2 s. For each time point, all elements on the video editing timeline are traversed to determine whether they are between the in-point and out-point of the element. If so, the count is incremented by 1. The total number of elements at the current time is placed in an array, and the number of element values included in the next time is calculated in sequence. An array arranged by time is obtained, recording the number of elements at each time point. When the creator adds a new video element, the video editing elements are detected. The element number array is updated.

[0079] S2: Starting from the starting point of the video editing timeline, a picture of the original video is intercepted every 0.2 s, and the picture error detection algorithm is executed on two adjacent pictures. The error values are placed in the error value array in chronological order. When entering the video editor, the picture error of the video is detected in a timely manner. After the creator adds or replaces a new video segment, the picture error degree of the new video segment is detected in a timely manner, and the error value array is updated.

[0080] S3: When the creator slides the timeline, obtain the current displayed time value t in milliseconds. Divide t by 200 and take the integer to get the index value. Retrieve the current number of time elements n from the above element quantity array. If n >= 3, then magnify the display of the timeline, horizontally magnify it by 2 times, so that the timeline elements can be seen more clearly. If n < 3, then retrieve the value w of the current image content error of the time from the error value array, and at the same time calculate the average value p of the image error array. If w / p >= 1.5, it is considered that the element change is large, then magnify the timeline, horizontally magnify it by 2 times. When it is less than 1.5, restore the normal timeline display.

[0081] In this exemplary application: (1) It is possible to dynamically adjust the zoom ratio of the timeline according to the number of elements (density) and the change range of the video content at the selected position (current displayed time) of the timeline, so as to achieve an optimized display of the video editing content, that is, to provide a more intuitive editing interface, enabling the editing personnel to locate and process video details faster, and improving the editing efficiency and editing experience. (2) It can automatically adapt to the editing requirements of different video scenarios without manual adjustment by the editing personnel, improving the flexibility and convenience of the editing process.

[0082] Embodiment 2 Figure 8 Schematically shows a block diagram of a video processing device according to Embodiment 2 of the present application. The device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. As Figure 8 shown, the device 1000 may include: a display module 1100, a first determination module 1200, a second determination module 1300, and an adjustment module 1400, where: The display module 1100 is used to display a video editing interface, and the video editing interface is used to display a timeline and a plurality of materials distributed on the timeline. Each material has a corresponding display area on the timeline; The first determination module 1200 is used to determine the current displayed time and the density and / or change range of the material corresponding to the current displayed time in response to an interaction operation on the timeline; wherein, the current displayed time is located within the display area of the corresponding material; The second determination module 1300 is used to determine the zoom ratio of the timeline according to the density and / or change range of the corresponding material; The adjustment module 1400 is used to adjust the timeline according to the zoom ratio to adjust the display area of the corresponding material.

[0083] As an alternative embodiment, the material includes elements; determining the density of the material corresponding to the current display time includes: Determining the number of elements corresponding to the current display time; wherein, the current display time is within the display area of the corresponding elements; Determining the density according to the number of the corresponding elements.

[0084] As an alternative embodiment, determining the number of elements corresponding to the current display time includes: Obtaining the number of the corresponding elements from a pre-constructed first array according to the current display time; wherein, the timeline includes multiple first time points, the first array includes the number of elements corresponding to each of the multiple first time points, and the current display time matches one of the first time points; Wherein, the first array is constructed through the following operations: Determining multiple first time points on the timeline according to a first preset time interval; For each first time point, determining the corresponding elements, and associating and writing the number of the corresponding elements with the first time point into the first array; wherein, the first time point is within the display area of the corresponding elements.

[0085] As an alternative embodiment, the video processing method further includes: Responding to a first editing operation to obtain the edited elements; wherein, the first editing operation includes adding and / or deleting elements; Updating the first array based on the display areas of the edited elements and the multiple first time points.

[0086] As an alternative embodiment, the material includes a video, the video includes multiple video frames; determining the change amplitude of the material corresponding to the current display time includes: Determining the image error between the current frame and the adjacent frame according to the current display time; wherein, there is a second preset time interval between the adjacent frame and the current frame and the adjacent frame is earlier than the current frame; Determining the change amplitude according to the image error between the current frame and the adjacent frame.

[0087] As an alternative embodiment, determining the image error between the current frame and the adjacent frame according to the current display time includes: Obtain the image error between the current frame and the adjacent frame from a pre-constructed second array according to the current display time; wherein, the timeline includes a plurality of second time points, the second array includes the image error between each second time point and the frame corresponding to the previous second time point, and the current display time matches one of the second time points; Wherein, the second array is constructed through the following operations: Determine a plurality of second time points on the timeline according to a second preset time interval; For each second time point, determine the image error between the corresponding frame and the corresponding frame of the previous second time point, and associate and write the image error with the second time point into the second array.

[0088] As an optional embodiment, the video processing method further includes: In response to a second editing operation, obtain the edited video; wherein, the second editing operation includes adding, deleting, and / or replacing videos; Update the second array based on multiple video frames of the edited video and the plurality of second time points.

[0089] As an optional embodiment, the material includes elements and videos, the video includes a plurality of video frames, the density of the corresponding material is the number of the corresponding elements, and the change range of the corresponding material is the change range of the current frame; Determine the scaling ratio of the timeline according to the density and / or change range of the corresponding material, including: Compare the number of the corresponding elements with a first preset threshold; When the number of the corresponding elements is greater than the first preset threshold, determine the magnification ratio of the timeline; When the number of the corresponding elements is less than the first preset threshold, determine the average change range according to the plurality of video frames; compare the ratio of the change range of the current frame to the average change range with a second preset threshold; when the ratio is greater than the second preset threshold, determine the magnification ratio of the timeline.

[0090] As an optional embodiment, adjust the timeline according to the scaling ratio to adjust the display area of the corresponding material, including: Magnify the timeline according to the magnification ratio to magnify the display area of the corresponding material; In response to an interaction operation on the magnified timeline, obtain the latest current display time and the density and / or change range of the latest corresponding material; When the density is less than the first preset threshold and the ratio of the change amplitude to the average change amplitude is less than the second preset threshold, determine the reduction ratio of the timeline; Reduce the timeline according to the reduction ratio to reduce the display area of the latest corresponding material.

[0091] Embodiment III Figure 9 Schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing the video processing method according to Embodiment III of the present application. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack-mounted server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers). As Figure 9 shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can be communicatively linked to each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium. The readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed on the computer device 10000, such as the program code of the video processing method. In addition, the memory 10010 may also be used to temporarily store various data that have been output or will be output.

[0092] The processor 10020 can be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other chips in some embodiments. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0093] The network interface 10030 can include a wireless network interface or a wired network interface. The network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network can be an enterprise internal network (Intranet), the Internet, the Global System of Mobile communication (abbreviated as GSM), Wideband Code Division Multiple Access (abbreviated as WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi and other wireless or wired networks.

[0094] It should be noted that Figure 9 Only the computer device with components 10010 - 10030 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0095] In this embodiment, the video processing method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as the processor 10020) to complete the embodiments of the present application.

[0096] Embodiment 4 The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video processing method in the embodiments are implemented.

[0097] In this embodiment, the computer-readable storage medium includes flash memory, hard disks, multimedia cards, card-type memories (such as SD or DX memories, etc.), random access memories (RAM), static random access memories (SRAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROM), programmable read-only memories (PROM), magnetic memories, magnetic disks, optical disks, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the computer device. Of course, the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device, such as the program code of the video processing method in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various data that have been output or are to be output.

[0098] Embodiment Five The embodiment of the present application further provides a computer program product, including a computer program, which when executed by a processor implements the method in the above embodiment.

[0099] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments of the present application can be implemented by a general-purpose computer device. They can be concentrated on a single computer device or distributed on a network composed of multiple computer devices. Optionally, they can be implemented by program codes executable by the computer device. Thus, they can be stored in a storage device and executed by the computer device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0100] It should be noted that the above are only the preferred embodiments of the present application, and do not limit the patent protection scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present application.

Claims

1. A video processing method, characterized in that, The method includes: Displaying a video editing interface for presenting a timeline and a plurality of materials distributed on the timeline, each material having a corresponding display area on the timeline; In response to an interaction operation on the timeline, determining a current display time and the density and / or change amplitude of the material corresponding to the current display time; wherein, the current display time is within the display area of the corresponding material; Determining a scaling ratio of the timeline according to the density and / or change amplitude of the corresponding material; Adjusting the timeline according to the scaling ratio to adjust the display area of the corresponding material.

2. The method according to claim 1, wherein The material includes elements; determining the density of the material corresponding to the current display time includes: Determining the number of elements corresponding to the current display time; wherein, the current display time is within the display area of the corresponding element; Determining the density according to the number of the corresponding elements.

3. The method according to claim 2, wherein Determining the number of elements corresponding to the current display time includes: Obtaining the number of the corresponding elements from a pre-constructed first array according to the current display time; wherein, the timeline includes a plurality of first time points, the first array includes the number of elements corresponding to each of the plurality of first time points, and the current display time matches one of the first time points; Wherein, the first array is constructed by the following operations: Determining a plurality of first time points on the timeline according to a first preset time interval; For each first time point, determining the corresponding element and associating and writing the number of the corresponding element with the first time point into the first array; wherein, the first time point is within the display area of the corresponding element.

4. The method according to claim 3, wherein The method further includes: In response to a first editing operation, obtaining the edited element; wherein, the first editing operation includes adding and / or deleting elements; Updating the first array based on the display area of the edited element and the plurality of first time points.

5. The method according to claim 1, wherein The material includes a video, and the video includes a plurality of video frames; Determining the change amplitude of the material corresponding to the current display time includes: Determining an image error between a current frame and an adjacent frame according to the current display time; wherein, the adjacent frame has a second preset time interval with the current frame and is earlier than the current frame; Determining the change amplitude according to the image error between the current frame and the adjacent frame.

6. The method according to claim 5, characterized in that, Determining an image error between a current frame and an adjacent frame according to the current display time includes: Obtaining the image error between the current frame and the adjacent frame from a pre-constructed second array according to the current display time; wherein, the timeline includes a plurality of second time points, the second array includes the image error between each second time point and the frame corresponding to the previous second time point, and the current display time matches one of the second time points; Wherein, the second array is constructed by the following operations: Determining a plurality of second time points on the timeline according to a second preset time interval; For each second time point, determine the image error between the corresponding frame and the corresponding frame of the previous second time point, and write the associated image error and the second time point into the second array.

7. The method according to claim 6, wherein The method further includes: In response to a second editing operation, obtain the edited video; wherein, the second editing operation includes adding, deleting, and / or replacing videos; Based on the multiple video frames of the edited video and the multiple second time points, update the second array.

8. The method according to claim 1, wherein The material includes elements and videos, the video includes multiple video frames, the density of the corresponding material is the number of the corresponding elements, and the change range of the corresponding material is the change range of the current frame; Determine the scaling ratio of the timeline according to the density and / or change range of the corresponding material, including: Compare the number of the corresponding elements with a first preset threshold; In the case where the number of the corresponding elements is greater than the first preset threshold, determine the magnification ratio of the timeline; In the case where the number of the corresponding elements is less than the first preset threshold, determine the average change range according to the multiple video frames; compare the ratio of the change range of the current frame to the average change range with a second preset threshold; in the case where the ratio is greater than the second preset threshold, determine the magnification ratio of the timeline.

9. The method according to claim 8, wherein Adjust the timeline according to the scaling ratio to adjust the display area of the corresponding material, including: Enlarge the timeline according to the magnification ratio to enlarge the display area of the corresponding material; In response to an interaction operation on the enlarged timeline, obtain the latest current display time and the density and / or change range of the latest corresponding material; In the case where the density is less than the first preset threshold and the ratio of the change range to the average change range is less than the second preset threshold, determine the reduction ratio of the timeline; Reduce the timeline according to the reduction ratio to reduce the display area of the latest corresponding material.

10. A video processing device, characterized in that, The device includes: A display module for displaying a video editing interface, the video editing interface being used to display a timeline and a plurality of materials distributed on the timeline, and each material has a corresponding display area on the timeline; A first determination module for, in response to an interaction operation on the timeline, determining the current display time and the density and / or change range of the material corresponding to the current display time; wherein, the current display time is located within the display area of the corresponding material; A second determination module for determining the scaling ratio of the timeline according to the density and / or change range of the corresponding material; An adjustment module for adjusting the timeline according to the scaling ratio to adjust the display area of the corresponding material.

11. A computer device, characterized in that, Includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein: The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to claims 1 to 9 are implemented.

Images