Methods, apparatus, devices, media, and products for processing video
Patent Information
- Application Number
- CN202610922033.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-24
- Publication Date
- 2026-09-22
Smart Images
Figure CN122802728A_ABST
Abstract
Description
Technical Field
[0001] This invention relates generally to the field of computers, and more specifically to methods, apparatus, devices, media, and products for processing video. Background Technology
[0002] With the continuous development of information technology and the widespread use of mobile terminals, video has become an important carrier for information transmission and communication, and the use of video editing applications for diversified video processing is becoming increasingly popular.
[0003] During the video editing process using related technologies, users can place materials such as videos, images, and stickers in specific areas of the video, such as playing specific video clips in the area of billboards that appear in the video. Summary of the Invention
[0004] In an example embodiment, a method, apparatus, device, medium, and product for processing video are provided.
[0005] In a first aspect, a method for processing video is provided. The method includes displaying a first processing result for the video, the first processing result indicating a region association effect of a first clip in the video. The method further includes, in response to a first operation on the first processing result, displaying a first keyframe of the video, the first keyframe including the first clip and a first video frame of the video. The method further includes, in response to a second operation on the first keyframe, displaying a second processing result for the video, the second operation indicating an adjustment to the region association effect of the first clip in the first video frame.
[0006] In a second aspect, an apparatus for processing video is provided, the apparatus comprising: a first display module configured to display a first processing result for the video, the first processing result indicating a region association effect of a first source material in the video; a second display module configured to display a first keyframe of the video in response to a first operation on the first processing result, the first keyframe including the first source material and a first video frame of the video; and a third display module configured to display a second processing result for the video in response to a second operation on the first keyframe, the second operation indicating an adjustment of the region association effect of the first source material in the first video frame.
[0007] In a third aspect, an electronic device is provided, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform a method for processing video. The method includes displaying a first processing result for the video, the first processing result indicating a region association effect of first material in the video. The method further includes, in response to a first operation on the first processing result, displaying a first keyframe of the video, the first keyframe including the first material and a first video frame of the video. The method further includes, in response to a second operation on the first keyframe, displaying a second processing result for the video, the second operation indicating an adjustment of the region association effect of the first material in the first video frame.
[0008] In a fourth aspect, a computer-readable storage medium is provided having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform a method for processing video. The method includes displaying a first processing result for the video, the first processing result indicating a region association effect of a first source material in the video. The method further includes displaying a first keyframe of the video, the first keyframe including the first source material and a first video frame, in response to a first operation on the first processing result. The method further includes displaying a second processing result for the video, in response to a second operation on the first keyframe, the second operation indicating an adjustment of the region association effect of the first source material in the first video frame.
[0009] In a fifth aspect, a computer program product is provided, having a computer program stored thereon that, when executed by a processor, implements a method for processing video. The method includes displaying a first processing result for the video, the first processing result indicating a region association effect of a first source material in the video. The method further includes, in response to a first operation on the first processing result, displaying a first keyframe of the video, the first keyframe including the first source material and a first video frame of the video. The method further includes, in response to a second operation on the first keyframe, displaying a second processing result for the video, the second operation indicating an adjustment of the region association effect of the first source material in the first video frame.
[0010] The summary section is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features, nor is it intended to limit the scope. Other features will become readily apparent from the following description. Attached Figure Description
[0011] Figure 1 A schematic diagram of an example environment in which the embodiments can be implemented is shown; Figure 2A and Figure 2B A schematic diagram showing the results of video processing is shown; Figure 3 A flowchart of a method 300 for processing video is shown; Figure 4A and Figure 4B A schematic diagram showing the result of the second processing is shown; Figure 5A and Figure 5B A schematic diagram showing a keyframe identifier is provided. Figure 6 A schematic block diagram of an example device 600 according to some embodiments is shown; and Figure 7 A block diagram of an example device that can be used to implement the embodiments is shown.
[0012] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation
[0013] The names of messages or information exchanged between multiple devices in the embodiments are for illustrative purposes only and are not intended to limit the scope of these messages or information. It is understood that before using the technical solutions disclosed in the embodiments, users should be informed of the type, scope of use, and usage scenarios of the personal information involved, and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.
[0014] For example, upon receiving a user's active request, a prompt message can be sent to the user, explicitly informing them that the requested operation will require the acquisition and use of their personal information. This allows the user to choose whether to provide personal information to the software or hardware (electronic device, application, server, or storage medium) performing the operations described herein, based on the prompt message. As an optional but non-limiting implementation, the prompt message can be sent to the user in the form of a pop-up window, which can present the prompt message in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0015] It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation method. Other methods that comply with relevant laws and regulations may also be applied to the embodiments.
[0016] The various embodiments will now be described in more detail with reference to the accompanying drawings. While some embodiments are shown in the drawings, it should be understood that implementations can be achieved in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the technical content of this document. It should be understood that the drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection herein.
[0017] In the description of the embodiments, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.
[0018] In the video editing process using related technologies, users can place video clips, images, stickers, and other materials in specific areas of the video. For example, a specific video clip can be played in the area of an advertising billboard in the video, causing that specific video clip to move and undergo perspective distortion synchronously with the billboard area, thus presenting a visual effect of the connection between the specific video clip and the billboard area. However, if users are not satisfied with the effect of the material's connection within the video after placing it in a specific area, they can only remove the material and replace it in the same area, which is resource-intensive and inefficient.
[0019] To address this, a method for video processing is provided, offering an interactive approach for adjusting the region association effect of footage within a video based on keyframes. After a user associates a piece of footage with a specific region of the video, if they are not satisfied with the region association effect, they can specifically adjust the footage in the keyframes to improve it. This method significantly saves resource consumption and improves video processing efficiency. Furthermore, it allows users to precisely adjust specific areas, enhancing the user experience.
[0020] The embodiments will now be described in further detail with reference to the accompanying drawings, wherein... Figure 1A schematic diagram of an example environment 100 in which embodiments can be implemented is shown. This example environment 100 includes a user device on which an application 110 (such as a media application supporting video editing, a video creation application, a video editing application, a social application supporting video editing, etc., but not limited thereto) can be installed. This application 110 serves as the core execution carrier of the methods in the embodiments of this application, undertaking tasks such as receiving video, editing video, displaying video, and responding to user interactions. In some embodiments, the user device is communicatively connected to a server, which provides background services for the application 110 running on the user device. For example, the server deploys a model (such as a machine learning model, but not limited thereto). The application 110 on the user device can communicate with the server to invoke the model on the server to perform various editing operations on the video (such as planar tracking, motion tracking, smoothing, etc.), receive the video processing results from the server, and display the video processing results for the user to view. Alternatively, the application 110 can set up an intelligent agent that can invoke the model on the server to perform various editing operations on the video.
[0021] For example, such as Figure 1 As shown, the user device displays the interface of application 110, on which the user can interact to edit the video. This interface is also called the video editing interface. For example, the user can upload a video to be edited (such as uploading a video from local storage) or select a video to be edited from the media library, and this is not limited to these. Furthermore, the user can select media from the media library to associate the media with specific areas of the video. Specifically, the user can place the media in a specific area of the video; for example, if the media is a sticker, the user can place the sticker in the billboard area of the video. The user can also attach the media to a specific area of the video. For example, if the media is the text "XYZ", the "Z" in the text can be attached to a specific corner of the video. It should be understood that this is merely illustrative and does not constitute a limitation of this application. In some embodiments, the process of associating media with specific areas of the video is also called planar tracking of the video. Planar tracking (also known as planar image tracking, planar motion matching, etc., but not limited to these) refers to tracking a flat surface (i.e., a specific area of the video) in a video frame, placing the material onto this plane, and having it move, rotate, and undergo perspective distortion synchronously with the plane to present the visual effect of the relationship between the material and the specific plane, that is, to present the effect of the material's regional relationship in the video. The material can be in the form of video clips, stickers, text, images, or GIFs, etc., and the embodiments of this application are not limited to this.
[0022] For example, the interface of application 110 includes a first interactive area 111 and a second interactive area 112. Application 110 can display the video the user wants to edit and a first material 113 to be associated with a specific area of the video in the first interactive area 111 (the first material 113 is shown as a sticker in the figure and does not constitute a limitation on the embodiments of this application). Application 110 can also display various editing items provided for the video in the second interactive area 112, such as stickers, animations, and tracking. Taking the editing item corresponding to planar tracking as an example, application 110 can display a planar tracking control 114 in the second interactive area 112. In response to a trigger operation on the planar tracking control 114, the first material 113 is associated with a specific area of the video to place the first material 113 on a specified plane of the video, thereby displaying a first processing result for the video, i.e., realizing planar tracking for the video. For example, the first processing result is displayed in the first interactive area 111, indicating the area association effect of the first material 113 in the video. Users can view the first processing result through the first interactive area 111, that is, view the regional association effect of the first material 113 in the video.
[0023] In some embodiments, the interface of application 110 also includes other interactive areas to provide more editing functions for the video. For example, the interface of application 110 also includes a third interactive area 115. Application 110 may display a timeline and at least one editing track in the third interactive area 115, wherein the editing track includes, but is not limited to: an editing track for the first material 113 (e.g., named the material editing track), an editing track for the video, an editing track for the audio, or an editing track for text (e.g., subtitles added to the video), etc. Exemplarily, a positioning component on at least one editing track can indicate the position of the video frame displayed in the first interactive area 111 within the entire video. Users can browse the video by clicking on the corresponding position on the editing track or by dragging the positioning component.
[0024] In the embodiments of this application, for Figure 1 The video editing scenario shown provides an interactive method for adjusting the region association effect of footage within a video based on keyframes. After a user associates a piece of footage with a specific region of the video, if they are not satisfied with the region association effect, they can specifically adjust the footage in the keyframes to improve the region association effect. For example, see below... Figure 2A and Figure 2B Let me introduce this process. Among them... Figure 2A and Figure 2B A schematic diagram showing the results of video processing is shown.
[0025] like Figure 2AAs shown, taking the example of a user associating the first material 113 with a specific area in a video, application 110 displays the first processing result for the video on the interface, for example, displaying the first processing result in the first interactive area 111. Based on this, application 110 provides a keyframe adjustment function for the first processing result. This keyframe adjustment function means that the user can adjust a certain keyframe in the video of the first processing result to improve the area association effect of the first material in that keyframe. For example, application 110 can display a first control 201 in the second interactive area 112. The first control 201 indicates adjusting the angle of the first material 113 to update the area association effect of the first material 113 in the keyframe of the video. For example, when the user is viewing the first processing result, and finds that the area association effect of the first material 113 in the video is poor at the 15-second mark, the user can click the first control 201 to trigger the keyframe adjustment function.
[0026] like Figure 2BAs shown, in response to a trigger operation on the first control 201, application 110 displays a first keyframe 202 of the video. This first keyframe 202 includes the first material 113 and the first video frame (in the figure, the first video frame is the video frame corresponding to the 15th second). In some embodiments, application 110 displays the first keyframe 202 in the first interactive area 111, where the first material 113 is superimposed on the first video frame, and the angle of the first material 113 is adjustable. That is, the user can trigger the first material 113 in the first interactive area 111 to adjust the angle of the first material 113 in the first video frame, thereby improving the regional correlation effect of the first material 113 in the first video frame. For example, the user can rotate the first material 113 to make the rotated first material 113 fit more closely to the tracking plane of the first video frame. Alternatively, the user can drag the lower left corner of the first material 113 to adjust its angle, making the adjusted first material 113 fit more closely to the tracking plane of the first video frame. After the user adjusts the first material 113, the application 110 can display a second processing result for the video. In some embodiments, the application 110 can also display at least one adjustment item 203 for the first material 113 (such as parameter adjustment items for the four corner coordinates of the first material 113, including upper left, upper right, lower left, and lower right) in the second interaction area 112. This adjustment item 203 indicates the input of coordinate parameters to adjust the angle of the first material 113. That is, the user can perform a trigger operation on at least one adjustment item 203 in the second interaction area 112 (such as manually inputting precise parameters or inputting precise parameters by adjusting the slider) to adjust the parameters related to the angle of the first material 113, thereby precisely controlling the angle of the first material 113. After the user adjusts the first material 113, the application 110 can display a second processing result for the video. In this way, the user can either directly operate on the first material 113 whose angle is adjustable in the first interaction area 111, or precisely adjust the angle of the first material 113 in the second interaction area 112, improving the flexibility of the solution.
[0027] For example, a triggering operation performed by a user on a first material 113 in the first interaction area 111 is associated with coordinate parameters displayed in at least one adjustment item 203 in the second interaction area 112. For instance, when a user drags the first material 113 in the first interaction area 111, the application can display the updated coordinate parameters of the first material 113 resulting from the user's dragging in the adjustment item 203. As another example, when a user inputs coordinate parameters in at least one adjustment item 203 in the second interaction area 112, the application can display the shape of the first material 113 corresponding to those coordinate parameters in the first interaction area 111. For example, in response to a triggering operation on the first material in the first interaction area, the application displays the coordinate parameters of the first material in at least one adjustment item in the second interaction area, wherein the coordinate parameters of the first material are associated with the triggering operation on the first material. Alternatively, in response to a triggering operation on at least one adjustment item in the second interaction area, the application displays an adjusted first material in the first interaction area, wherein the angle of the adjusted first material is associated with the triggering operation on at least one adjustment item.
[0028] Combination Figure 2A and Figure 2B As shown in the process, users can improve the overall region association effect by simply adjusting keyframes without re-associating the footage with specific regions of the video. This significantly reduces resource consumption and improves the intuitiveness and accuracy of the adjustments.
[0029] The user equipment described above can be any type of mobile computing device, including mobile computers (e.g., personal digital assistants, laptops, notebooks, tablets, netbooks, etc.), mobile phones (e.g., cellular phones, smartphones, etc.), wearable computing devices (e.g., smartwatches, head-mounted devices, including smart glasses, etc.) or other types of mobile devices. In some embodiments, the user equipment can also be a fixed computing device, such as a desktop computer, game console, smart TV, etc. It should be understood that the above... Figures 1 to 2B The interface layout of application 110 shown is merely an illustrative example and does not constitute a limitation of this application. For example, when application 110 runs on a mobile device with a small screen size, such as a smartphone, the above-mentioned... Figure 1The interface layout of the first interactive area 111, the second interactive area 112, and the third interactive area 115 shown can be adaptively adjusted according to the screen size and interaction method of the device. For example, on a mobile device, the first interactive area 111 can occupy the main display area of the screen, while the second interactive area 112 and the third interactive area 115 can be presented through bottom, side sliding panels, or folded menus, thereby achieving complete video editing functions within a limited screen space. In this way, the video processing method provided in this application embodiment can adapt to various types of user devices, and has good cross-platform applicability and interactive flexibility.
[0030] It should be understood that the architecture and functionality in the example environment are described for illustrative purposes only and do not imply any limitation on the scope. The embodiments can also be applied to other environments with different architectures and / or functionalities.
[0031] The process of the embodiments will be described in detail below with reference to other accompanying drawings. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and their scope is not limited in this respect.
[0032] Figure 3 A flowchart of a method 300 for processing video is shown. In this embodiment, it can be performed by... Figure 1 The method is executed by application 110 on a user device in an embodiment. In block 302, a first processing result for the video is displayed, indicating the region association effect of the first material in the video. The first processing result is obtained based on the association of the first material with a specific region in the video. Exemplarily, the application can communicate with a server to perform planar tracking for the video. For example, in response to a planar tracking operation performed by the user on the video, the application sends the first material and the video to the server. The server processes the first material and the video according to the tracking plane indicated by the planar tracking operation to obtain the first processing result and returns the first processing result to the application. The application displays the first processing result on the interface for the user to view.
[0033] In box 304, in response to a first operation on the first processing result, a first keyframe of the video is displayed. The first keyframe includes the first source material and the first video frame. The first operation refers to a trigger operation for the keyframe adjustment function, which can be a trigger operation for a control (such as...). Figure 2A and Figure 2BThe first control 201 shown is the "keyframe adjustment" control. The first operation can also be user voice input (such as the user saying "adjust the tracking effect of the current frame" or "start keyframe adjustment," etc., and is not limited thereto), or other shortcut operations implemented on the interface. The embodiments of this application do not limit this. For example, taking a trigger operation on a control as an example, the application can display a first control that indicates adjusting the angle of the first material to update the regional correlation effect of the first material in the keyframes of the video. Accordingly, the application displays a first keyframe in response to the first trigger operation on the first control. The first trigger operation is, for example, a click operation. For instance, while browsing the first processing result, the user finds that the regional correlation effect at 15 seconds is poor and locates it at 15 seconds. Based on this, the application, in response to the first trigger operation on the first control, converts the video frame at 15 seconds into a first keyframe.
[0034] In some embodiments, the application may also display a first prompt message in response to a second triggering operation on the first control. The first prompt message indicates the function of the first control, such as "Adjustment only applies to keyframes" or "Enter keyframe adjustment mode," etc., and is not limited thereto. The second triggering operation is, for example, when the user operates the application with a mouse and moves the mouse to the first control. Alternatively, the second triggering operation may be a click operation different from the first triggering operation, such as the first triggering operation being a long press and the second triggering operation being a single click, etc., and is not limited thereto.
[0035] In some embodiments, the application displays a first keyframe in a first interactive area, wherein first footage is overlaid on the first video frame, and the angle of the first footage is adjustable. For example, see reference. Figure 2B The first keyframe 202 shown here has the first clip 113 superimposed on the first video frame, and the angle of the first clip 113 is adjustable. In this way, the scope of operation when adjusting the first keyframe can be clearly defined, and by showing that the angle of the first clip is adjustable, the interactive prompt "modification is possible" can be clearly conveyed to the user, reducing the user's cognitive load.
[0036] In some embodiments, based on displaying a first keyframe in a first interactive area, the application displays at least one edit track in a third interactive area. Positioning components on the at least one edit track indicate the position of the first video frame in the video. Based on this, a user can trigger an operation on the at least one edit track to browse the video. For example, in response to a trigger operation on the at least one edit track, the application displays frames from the video other than the first video frame in the first interactive area.
[0037] In box 306, in response to a second operation on the first keyframe, a second processing result for the video is displayed. The second operation indicates adjusting the region association effect of the first clip in the first video frame. The second operation refers to a triggering operation that adjusts the angle of the first clip in the first keyframe. After the user performs the second operation on the first keyframe, the application displays the second processing result. It should be understood that after the user adjusts the keyframe, the application does not modify the existing tracking plane data, and subsequently added clips can still be associated with the existing plane data. In some embodiments, the second operation is a triggering operation on the first clip. Exemplarily, the application displays the second processing result in response to a triggering operation on the first clip in a first interactive area. For example, refer to... Figure 2B The first material 113 shown is angle-adjustable, allowing the user to drag or rotate it to perform a second operation on the first keyframe. In other embodiments, the second operation is a trigger operation for at least one adjustment item. For example, in response to the first operation, the application displays at least one adjustment item for the first material in a second interaction area, indicating an adjustment to the angle of the first material. Based on this, the application displays a second processing result in response to a trigger operation for at least one adjustment item in the second interaction area. For example, refer to... Figure 2B The user can trigger an operation on at least one adjustment item 203 to adjust parameters related to the angle of the first material 113, thereby precisely controlling the angle of the first material 113 and realizing a second operation for the first keyframe.
[0038] In some embodiments, the second processing result is the result after smoothing. It should be understood that in scenarios where the user only adjusts the first keyframe, the application can perform smoothing processing on video frames near the adjusted first keyframe. That is, smoothing processing is performed on the motion trajectory of the first material in the first video frame and other video frames. For example, smoothing processing algorithms are applied to the trajectory data of the first material in each video frame, such as position parameters, rotation angle parameters, or perspective distortion parameters, to make the motion of the first material in the video smoother and more natural.
[0039] The above Figure 3The method 300 shown provides an interactive way to adjust the regional association effect of footage in a video based on keyframes for video editing scenarios. After a user associates a piece of footage with a specific region of the video, if they are not satisfied with the regional association effect, they can specifically adjust the footage in the keyframes to improve the regional association effect. That is, users can improve the overall regional association effect by only adjusting keyframes without re-associating the footage with a specific region of the video, thus significantly reducing resource consumption and improving the intuitiveness and accuracy of the adjustments.
[0040] In some embodiments, in addition to the above-described interaction methods, the application also supports more functions related to keyframe adjustment to further improve human-computer interaction efficiency and enhance user experience. For example, besides supporting keyframe adjustment, the application also supports global adjustment. Global adjustment means that when the angle of the first clip is adjusted, this adjustment will be reflected in all frames in the video involving the first clip. For instance, the application displays a first control and a second control, where the first control indicates adjusting the angle of the first clip to update the regional association effect of the first clip in the video's keyframes, and the second control indicates adjusting the angle of the first clip to update the global regional association effect of the first clip in the video. That is, the user can select different adjustment functions to adjust the first clip based on the first processing result.
[0041] In some embodiments, after the user completes the adjustment of the first keyframe using the method 300 described above, the application displays the second processing result. In this case, if the user needs to use the global adjustment function, the application can display a second prompt message to indicate whether the user needs to retain the second processing result. For example, Figure 4A and Figure 4B A schematic diagram showing the result of the second processing is illustrated. For example... Figure 4A As shown, after the user completes the adjustment of the first keyframe using the method 300 described above, application 110 displays the second processing result. In the second processing result, the first material 113 on the first keyframe has been adjusted to better fit the tracking plane in the video. Based on this, the user can trigger an operation on the second control 204 (such as the "Global Adjustment" control) to use the global adjustment function. Figure 4BAs shown, in response to a trigger operation on the second control 204, application 110 displays a second prompt message 205. This second prompt message 205 indicates the user's confirmation of the retention mode for the second processing result, preventing accidental data loss due to user error. For example, the second prompt message 205 could be "Switching will result in the loss of adjusted keyframes. Are you sure you want to switch?" Application 110 can also display "Confirm Switch" and "Do Not Switch" options on the second prompt message 205 for the user to choose from. It should be noted that the content of the second prompt message 205 is merely illustrative and does not constitute a limitation on the embodiments of this application. In some embodiments, the second prompt message could also be "Please confirm whether to switch to global adjustment while retaining the adjusted keyframes." That is, application 110 supports switching to global adjustment while retaining the adjustment data for the first keyframe, allowing the user to perform global adjustments on video frames other than the first keyframe in the second processing result.
[0042] In some embodiments, the application may display keyframe markers on the footage editing track for the first clip to indicate the first keyframe, making it easier for the user to understand the location of the first keyframe within the entire video. For example, Figure 5A and Figure 5B A schematic diagram showing keyframe identifiers is illustrated. For example... Figure 5A As shown, application 110 displays a material editing track 501 for the first material 113 in the third interaction area 115, and displays a keyframe identifier 502 (e.g., named the first keyframe identifier) corresponding to the first keyframe 202 on the material editing track 501. In some embodiments, when more than one keyframe needs to be adjusted, the application can display at least one keyframe identifier on the material editing track 501 according to the number and position of these keyframes. For example, as Figure 5B As shown, application 110 can display at least one keyframe identifier 502 on the media editing track 501, wherein the at least one keyframe identifier includes the first keyframe identifier corresponding to the first keyframe 202. In this way, users can intuitively see which positions of the current first media 113 involve keyframes on the video timeline, thus facilitating quick location and navigation.
[0043] In some embodiments, the application also provides an operation method for managing keyframes based on the media editing track. For example, a user can perform at least one trigger operation on keyframe identifiers on the media editing track to achieve management functions such as adding, deleting, and moving corresponding keyframes. For example, in response to a user's first trigger operation on the media editing track, the application de-displays a second keyframe identifier from at least one keyframe identifier on the editing track to indicate the deletion of the second keyframe of the video. For instance, a user can click on a second keyframe identifier, triggering the application to display at least one operation item for the second keyframe identifier. In response to the user's trigger operation on the "delete" operation item, the application de-displays the second keyframe identifier to indicate the deletion of the second keyframe of the video. For example, in response to a second trigger operation on the media editing track, the application adds a third keyframe identifier to the media editing track to indicate the addition of a third keyframe of the video. For instance, a user can click on a specified position on the media editing track to convert the video frame corresponding to that specified position into a third keyframe; accordingly, the application adds a third keyframe identifier to the media editing track. For example, in response to a third trigger operation on the media editing track, the application displays the movement trajectory of a fourth keyframe identifier among at least one keyframe identifier on the media editing track to indicate the position of the fourth keyframe in the video to be adjusted. For instance, a user can drag the fourth keyframe identifier on the editing track, triggering the application to display the movement trajectory of that fourth keyframe identifier on the media editing track. For example, the fourth keyframe could be adjusted from the 10th second to the 12th second. In this way, users can perform comprehensive management of keyframes on the media editing track without switching interfaces, significantly improving the efficiency and intuitiveness of keyframe adjustments.
[0044] In some embodiments, the application also supports keyframe switching via keyframe markers. For example, in response to a user's trigger action on the fifth keyframe marker among at least one keyframe markers (e.g., clicking the fifth keyframe marker), the application switches the currently displayed first keyframe to the fifth keyframe of the video. That is, the user can quickly jump to the corresponding keyframe screen by clicking any keyframe marker on the editing track and continue using the aforementioned keyframe adjustment functions on that screen. This allows users to quickly switch between multiple keyframes, facilitating the correction of tracking deviations occurring at multiple locations in the video and effectively improving processing efficiency in multi-keyframe scenarios.
[0045] In some embodiments, the application can display at least one keyframe marker on the editing track at various triggering times. For example, the application may display the first keyframe marker on the editing track in response to the aforementioned first operation (i.e., the operation that triggers the keyframe adjustment function). That is, when the user wants to trigger the keyframe adjustment function for a certain frame, the application immediately displays the corresponding keyframe marker on the editing track. Alternatively, the application may display at least one keyframe marker on the editing track in response to displaying the first processing result. That is, after completing the initial planar tracking of the video, the distribution of keyframes in the video is automatically displayed on the editing track. These multiple triggering methods provide users with flexible entry points, improving the convenience and adaptability of the interaction.
[0046] In some embodiments, when a user associates multiple clips with specific areas of a video, the application also supports keyframe adjustment functionality for multi-clip scenarios. For example, the first processing result can also indicate the area association effect of the second clip in the video, and the first and second clips overlap in the video. When the user adjusts the first clip in the aforementioned manner, the adjustment of the first clip may affect the area association effect of the second clip in the overlapping area because the first and second clips overlap. Based on this, the application can display a third prompt message, indicating that the overlapping area corresponding to the second clip should be adjusted according to the second operation (i.e., the user's adjustment operation on the first clip). For example, the third prompt message can prompt the user, "Other clips have been detected overlapping with the currently adjusted area. Do you want to adjust the overlapping area synchronously?". Based on this, in response to the user's trigger operation on the third prompt message (e.g., clicking the "Confirm Synchronous Adjustment" button), the application displays a third processing result for the video, which is the result obtained by further synchronously adjusting the overlapping area of the second clip according to the same adjustment method as the first clip, based on the second processing result. In this way, when a user makes keyframe adjustments to a clip, if the adjusted area overlaps with other clips, the application can promptly notify the user and process the overlapping area synchronously, further improving the human-computer interaction efficiency for adjusting area correlation effects.
[0047] In some embodiments, after a user associates footage with a specific region of a video, the application also supports re-tracking video segments within the video. This provides a local re-tracking function. For example, after the application displays the first processing result (i.e., the initial planar tracking result), if the user is dissatisfied with the region association effect within a certain video segment, they can perform local re-tracking on that video segment. For example, refer to... Figure 2A and Figure 2BUsers can first select a video frame (e.g., frame 10) on the editing track in the third interactive area 115, and then click the "Tracking Direction" control in the second interactive area 112 to trigger the application to display multiple tracking options, such as "Tracking to the Right," "Tracking to the Left," and "Bidirectional Tracking." Users can choose "Tracking to the Right" to perform local retracking. Based on this, application 110 can only re-implement planar tracking for video frames from frame 10 onwards, while the tracking data before frame 10 remains unchanged. That is, local retracking only replaces the corresponding tracking data within the user-specified video segment, without clearing all existing tracking data. In this way, users do not need to re-associate the material with a specific area of the complete video every time they make an adjustment, but can only make local corrections to unsatisfactory video segments, thereby significantly reducing resource consumption while retaining existing effective tracking data and further improving the efficiency of video editing.
[0048] In some embodiments, after the user associates the footage with a specific region of the video and completes the adjustment of the first keyframe using the method 300 described above, the application displays a second processing result. During this process, the application can communicate with the server to generate the second processing result. For example, the application can capture a partial image of the first keyframe based on the position of the first footage within the first keyframe and send the partial image to the server to perform processing related to planar tracking. For instance, by extending a preset distance outward from the area occupied by the first footage in the first keyframe, a partial image of the first keyframe can be obtained. This significantly reduces the amount of data transmitted from the application to the server, effectively reducing the server's processing load and thereby improving the response speed and overall processing efficiency of the video processing.
[0049] Figure 6 A schematic block diagram of an example device 600 according to some embodiments is shown. Device 600 can be implemented in software, hardware, or a combination of both. Figure 6 As shown, the device 600 includes a first display module 601, a second display module 602, and a third display module 603. The first display module 601 is configured to display a first processing result for the video, indicating the region association effect of the first material in the video. The second display module 602 is configured to display a first keyframe of the video in response to a first operation on the first processing result, the first keyframe including the first material and a first video frame. The third display module 603 is configured to display a second processing result for the video in response to a second operation on the first keyframe, the second operation indicating an adjustment to the region association effect of the first material in the first video frame.
[0050] In some embodiments, the second display module 602 is configured to display a first keyframe in a first interactive area, wherein the first material is overlaid on the first video frame, and the angle of the first material is adjustable.
[0051] In some embodiments, the third display module 603 is configured to display a second processing result in response to a trigger operation on the first material in the first interactive area.
[0052] In some embodiments, the device 600 further includes a fourth display module configured to display at least one adjustment item for the first material in a second interactive area in response to a first operation, the adjustment item indicating input coordinate parameters to adjust the angle of the first material.
[0053] In some embodiments, the third display module 603 is configured to display a second processing result in response to a triggering operation for at least one adjustment item in the second interaction area.
[0054] In some embodiments, the device 600 further includes a fifth display module configured to: display coordinate parameters of the first material in at least one adjustment item in a second interaction area in response to a trigger operation for the first material in a first interaction area, wherein the coordinate parameters of the first material are associated with the trigger operation for the first material; or, display an adjusted first material in a first interaction area in response to a trigger operation for at least one adjustment item in a second interaction area, wherein the angle of the adjusted first material is associated with the trigger operation for at least one adjustment item.
[0055] In some embodiments, the device 600 further includes: a sixth display module configured to display at least one editing track in a third interactive area, wherein a positioning component on the at least one editing track indicates the position of the first video frame in the video; and to display, in response to a triggering operation on the at least one editing track, frames in the video other than the first video frame in the first interactive area.
[0056] In some embodiments, at least one editing track includes a material editing track for the first material, and the sixth display module is further configured to display at least one keyframe identifier on the material editing track, the at least one keyframe identifier including a first keyframe identifier corresponding to the first keyframe.
[0057] In some embodiments, the sixth display module is further configured to: in response to a first trigger operation on the media editing track, de-display a second keyframe identifier among at least one keyframe identifier on the media editing track to indicate the deletion of a second keyframe of the video; in response to a second trigger operation on the media editing track, add a third keyframe identifier on the media editing track to indicate the addition of a third keyframe of the video; or in response to a third trigger operation on the media editing track, display the movement trajectory of a fourth keyframe identifier among at least one keyframe identifier on the media editing track to indicate the adjustment of the position of the fourth keyframe of the video.
[0058] In some embodiments, the apparatus 600 further includes a seventh display module configured to switch the first keyframe to be displayed as the fifth keyframe of the video in response to a trigger operation on a fifth keyframe identifier among at least one keyframe identifier.
[0059] In some embodiments, the sixth display module is further configured to display a first keyframe identifier on the material editing track in response to the first operation.
[0060] In some embodiments, the device 600 further includes: an eighth display module configured to display a first control and a second control, wherein the first control indicates adjusting the angle of the first clip to update the region association effect of the first clip in a keyframe of the video, and the second control indicates adjusting the angle of the first clip to update the global region association effect of the first clip in the video. A second display module 602 is configured to display a first keyframe in response to a first trigger operation on the first control.
[0061] In some embodiments, the device 600 further includes a ninth display module configured to display a first prompt message in response to a second trigger operation on the first control, the first prompt message indicating the function of the first control.
[0062] In some embodiments, the device 600 further includes a tenth display module configured to display a second prompt message in response to a triggering operation on the second control after displaying the second processing result, the second prompt message indicating confirmation of the retention mode of the second processing result.
[0063] In some embodiments, the first processing result further indicates the regional association effect of the second material in the video and that the first material and the second material have overlapping areas; the device 600 further includes: an eleventh display module configured to display third prompt information, the third prompt information indicating that the overlapping area corresponding to the second material is adjusted according to the second operation; and in response to a triggering operation for the third prompt information, displaying a third processing result for the video.
[0064] This device provides an interactive method for adjusting the regional association effect of footage within a video based on keyframes, specifically for video editing scenarios. After a user associates a piece of footage with a specific region of the video, if they are not satisfied with the regional association effect, they can selectively adjust the footage in the keyframes to improve the regional association effect. In other words, users can improve the overall regional association effect by locally adjusting keyframes without reassigning the footage to a specific region of the video, thus significantly reducing resource consumption and improving the intuitiveness and accuracy of adjustments.
[0065] The division of modules or units in the embodiments of this article is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. In addition, the functional units in the embodiments of this article can be integrated into one unit, exist as separate physical entities, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0066] Figure 7 A block diagram of an example device 700 that can be used to implement the embodiments is shown. It should be understood that... Figure 7 The device 700 shown is merely an example and should not be construed as limiting the functionality and scope of the implementation described herein. For example, device 700 may correspond to the implementation described herein. Figure 1 The user equipment described above can be used to perform the above-described... Figures 1 to 5B The process. For example, device 700 may correspond to the electronic device of the third aspect of the invention.
[0067] like Figure 7 As shown, device 700 is in the form of a general-purpose computing device. Components of device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of device 700.
[0068] Device 700 typically includes multiple computer storage media. Such media can be any available media accessible to device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 730 can be removable or non-removable media and may include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within device 700.
[0069] Device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not specified in the original text... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions in various implementations.
[0070] The communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of device 700 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.
[0071] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with device 700, or with any device that enables device 700 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).
[0072] Furthermore, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. This computer-readable storage medium may be a non-transitory computer-readable storage medium. Furthermore, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the method described above. Additionally, a computer program product is provided that stores a computer program thereon, which, when executed by a processor, implements the method described above.
[0073] Various aspects of the situation are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to various embodiments. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0074] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0075] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0076] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products based on multiple implementations. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0077] The implementations have been described above. The foregoing descriptions are illustrative and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.
Claims
1. A method for processing video, comprising: Displays a first processing result for the video, the first processing result indicating the region association effect of the first material in the video; In response to a first operation on the first processing result, a first keyframe of the video is displayed, the first keyframe including the first material and a first video frame of the video; as well as In response to a second operation on the first keyframe, a second processing result for the video is displayed, the second operation indicating an adjustment to the region association effect of the first material in the first video frame.
2. The method according to claim 1, wherein displaying the first keyframe of the video comprises: The first keyframe is displayed in the first interactive area, wherein the first material is overlaid on the first video frame, and the angle of the first material is adjustable.
3. The method according to claim 2, wherein displaying the second processing result in response to a second operation on the first keyframe includes: In response to a trigger operation on the first material in the first interactive area, the second processing result is displayed.
4. The method according to claim 3, further comprising: In response to the first operation, at least one adjustment item for the first material is displayed in the second interactive area, the adjustment item indicating the input of coordinate parameters to adjust the angle of the first material.
5. The method of claim 4, wherein displaying the second processing result in response to a second operation on the first keyframe includes: In response to a triggered operation for the at least one adjustment item in the second interaction area, the second processing result is displayed.
6. The method of claim 4, further comprising one or more of the following: In response to a trigger operation on the first material in the first interactive area, coordinate parameters of the first material are displayed in at least one adjustment item in the second interactive area, wherein the coordinate parameters of the first material are associated with the trigger operation on the first material; or In response to a trigger operation for the at least one adjustment item in the second interaction area, the adjusted first material is displayed in the first interaction area, wherein the angle of the adjusted first material is associated with the trigger operation for the at least one adjustment item.
7. The method according to claim 2, further comprising: At least one editing track is displayed in the third interactive area, and positioning components on the at least one editing track indicate the position of the first video frame in the video; as well as In response to a trigger operation on the at least one edit track, the video frame other than the first video frame is displayed in the first interactive area.
8. The method of claim 7, wherein the at least one editing track includes a material editing track for the first material; further comprising: At least one keyframe identifier is displayed on the material editing track, and the at least one keyframe identifier includes the first keyframe identifier corresponding to the first keyframe.
9. The method of claim 8, further comprising any one of the following: In response to a first trigger operation on the media editing track, the second keyframe identifier among the at least one keyframe identifier is de-displayed on the media editing track to indicate the deletion of the second keyframe of the video; In response to a second trigger operation on the media editing track, a third keyframe identifier is added to the media editing track to indicate the addition of a third keyframe to the video. or In response to a third trigger operation on the media editing track, the movement trajectory of the fourth keyframe identifier among the at least one keyframe identifier is displayed on the media editing track to indicate the adjustment of the position of the fourth keyframe of the video.
10. The method of claim 8, further comprising: In response to a trigger operation targeting the fifth keyframe identifier among the at least one keyframe identifiers, the first keyframe is switched to be displayed as the fifth keyframe of the video.
11. The method of claim 8, wherein displaying at least one keyframe identifier on the material editing track comprises: In response to the first operation, the first keyframe identifier is displayed on the material editing track.
12. The method according to claim 1, further comprising: Display a first control and a second control, wherein the first control instructs to adjust the angle of the first material to update the region association effect of the first material in the keyframe of the video, and the second control instructs to adjust the angle of the first material to update the global region association effect of the first material in the video; The step of displaying a first keyframe of the video in response to a first operation on the first processing result includes: displaying the first keyframe in response to a first trigger operation on the first control.
13. The method of claim 12, further comprising: In response to a second triggering operation on the first control, a first prompt message is displayed, the first prompt message indicating the function of the first control.
14. The method of claim 12, further comprising: In response to a triggering operation on the second control after the second processing result is displayed, a second prompt message is displayed, indicating confirmation of the retention mode for the second processing result.
15. The method of claim 1, wherein the first processing result further indicates a region association effect of the second material in the video and that the first material and the second material have overlapping regions; further comprising: A third prompt message is displayed, indicating that the overlapping area corresponding to the second material should be adjusted according to the second operation; as well as In response to the triggering operation of the third prompt information, the third processing result for the video is displayed.
16. An apparatus for processing video, comprising: The first display module is configured to display a first processing result for the video, wherein the first processing result indicates the regional association effect of the first material in the video; The second display module is configured to display a first keyframe of the video in response to a first operation on the first processing result, the first keyframe including the first material and the first video frame of the video. as well as The third display module is configured to display a second processing result for the video in response to a second operation on the first keyframe, wherein the second operation indicates an adjustment of the region association effect of the first material in the first video frame.
17. The apparatus according to claim 16, wherein the second display module is configured as: The first keyframe is displayed in the first interactive area, wherein the first material is overlaid on the first video frame, and the angle of the first material is adjustable.
18. An electronic device comprising: At least one processing unit; At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 15 when executed by the at least one processing unit.
19. A computer-readable storage medium having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 15.
20. A computer program product having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 15.