Video processing method, video processing device, and video processing system

Abstract

Provided in the present disclosure are a video processing method, a video processing device, and a video processing system, which may be applied to the technical field of video processing. The video processing method comprises: in response to a stabilization mode switching instruction, processing, in a second stabilization mode, a second video clip of a video to be processed, so as to obtain a first target video clip, wherein the video to be processed further comprises a first video clip, the first video clip has a stabilization effect corresponding to a first stabilization mode, and the first stabilization mode is different from the second stabilization mode.

Description

Video processing methods, video processing equipment, video processing systems Technical Field

[0001] This disclosure relates to the field of video processing technology, and more specifically, to a video processing method, video processing equipment, and video processing system. Background Technology

[0002] In the field of video shooting, video stabilization technology can effectively reduce image shake caused by factors such as equipment shaking, hand tremors, or movement during shooting, thereby significantly improving video stability and user viewing experience, and thus plays a vital role in video processing.

[0003] However, although video stabilization technology improves the stability of video footage, it can only apply the same type of stabilization to the entire video, and cannot be flexibly adjusted according to different scenarios and user needs. In other words, it cannot achieve segmented and differentiated stabilization processing for the entire video. Summary of the Invention

[0004] In view of this, the present disclosure provides a video processing method, a video processing device, and a video processing system.

[0005] According to one aspect of this disclosure, a video processing method is provided, characterized in that the method includes: in response to a stabilization mode switching command, processing a second video segment of a video to be processed using a second stabilization mode to obtain a first target video segment; wherein the video to be processed further includes a first video segment, the first video segment having a stabilization effect corresponding to a first stabilization mode, and the first stabilization mode being different from the second stabilization mode.

[0006] According to another aspect of this disclosure, a video processing apparatus is provided, comprising: one or more processors, operating individually or jointly; and a memory communicatively connected to the processors; wherein the memory stores instructions executable by the processors, the instructions being executed by the processors to cause the processors, in response to a stabilization mode switching instruction, to process a second video segment of a video to be processed using a second stabilization mode to obtain a first target video segment; wherein the video to be processed further includes the first video segment, the first video segment having a stabilization effect corresponding to a first stabilization mode, the first stabilization mode being different from the second stabilization mode.

[0007] According to another aspect of this disclosure, a video processing system is provided, comprising: a video acquisition device for acquiring a video to be processed; and a video processing device for processing a second video segment of the video to be processed in response to a stabilization mode switching command, using a second stabilization mode to obtain a first target video segment; wherein the video to be processed further includes a first video segment, the first video segment having a stabilization effect corresponding to a first stabilization mode, and the first stabilization mode being different from the second stabilization mode.

[0008] According to embodiments of this disclosure, by responding to a stabilization mode switching command, different stabilization modes can be applied to different video segments in the video being processed. This not only effectively reduces video shake but also avoids the monotony caused by using the same stabilization mode for the entire video, improving the flexibility and adaptability of video stabilization processing, thereby enhancing the stability and viewing experience of the entire video. Furthermore, the above method can also meet users' personalized needs for video stabilization in different scenarios, enhancing the video shooting and viewing experience. Attached Figure Description

[0009] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0010] Figure 1 schematically illustrates a system architecture to which video processing methods can be applied according to embodiments of the present disclosure;

[0011] Figure 2 schematically illustrates a flowchart of a video processing method according to an embodiment of the present disclosure;

[0012] Figure 3A schematically illustrates an example of a second video segment acquisition process according to an embodiment of the present disclosure;

[0013] Figure 3B schematically illustrates an example of a second video segment acquisition process according to another embodiment of the present disclosure;

[0014] Figure 4A schematically illustrates an example of the image stabilization effect of each video segment in a video to be processed according to an embodiment of the present disclosure;

[0015] Figure 4B schematically illustrates an example of the image stabilization effect of video segments in a video to be processed according to another embodiment of the present disclosure;

[0016] Figure 5A schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method to obtain a first target video segment according to an embodiment of the present disclosure.

[0017] Figure 5B schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method to obtain a first target video segment according to another embodiment of the present disclosure.

[0018] Figure 6A schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method to obtain a first target video segment according to yet another embodiment of the present disclosure.

[0019] Figure 6B schematically illustrates an example of the process of processing a second video segment of a video to be processed using a second stabilization method to obtain a first target video segment according to another embodiment of the present disclosure.

[0020] Figure 7A schematically illustrates an example of a process of processing a first target video segment based on a tracking operation to obtain a second target video segment according to an embodiment of the present disclosure;

[0021] Figure 7B schematically illustrates an example of a process of processing a first target video segment based on a dot-mapping operation to obtain a second target video segment according to an embodiment of the present disclosure.

[0022] Figure 7C schematically illustrates an example of a process of processing a first target video segment based on a forward direction selection operation to obtain a second target video segment according to an embodiment of the present disclosure;

[0023] Figure 7D schematically illustrates an example of a process of processing a first target video segment to obtain a second target video segment according to an embodiment of the present disclosure based on a free-viewpoint adjustment operation;

[0024] Figure 7E schematically illustrates an example of a process in which a first target video segment is processed simultaneously based on a tracking operation and a free-viewpoint adjustment operation according to an embodiment of the present disclosure to obtain a second target video segment.

[0025] Figure 7F schematically illustrates an example of the process of processing a first target video segment to obtain a second target video segment according to an embodiment of the present disclosure based on dot-mapping operation and free viewpoint adjustment operation;

[0026] Figure 7G schematically illustrates an example of the process of processing a first target video segment to obtain a second target video segment according to an embodiment of the present disclosure based on a forward direction selection operation and a free viewpoint adjustment operation;

[0027] Figure 8 schematically illustrates an example diagram of a video processing procedure according to an embodiment of the present disclosure;

[0028] Figure 9 schematically illustrates a block diagram of a video processing apparatus according to an embodiment of the present disclosure; and

[0029] Figure 10 schematically illustrates a video processing system according to an embodiment of the present disclosure. Detailed Implementation

[0030] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0033] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0034] In the technical solution disclosed herein, the acquisition, storage, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0035] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0036] In one example, when faced with certain dynamic scenes, users actually expect a portion of the video to change perspective appropriately with movement to enhance immersion, while overly stable footage appears static, lacking realism and dynamic effects.

[0037] Therefore, this embodiment of the disclosure, by responding to a stabilization mode switching command, can apply different stabilization modes to different video segments in the video being processed. This not only effectively reduces video shake but also avoids the monotony caused by using the same stabilization mode for the entire video, improving the flexibility and adaptability of video stabilization processing, thereby enhancing the stability and viewing experience of the entire video. Furthermore, the above method can also meet users' personalized needs for video stabilization in different scenarios, enhancing the video shooting and viewing experience.

[0038] Figure 1 schematically illustrates a system architecture to which video processing methods can be applied according to embodiments of the present disclosure. It should be noted that Figure 1 is merely an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not imply that embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenarios.

[0039] As shown in Figure 1, the system architecture 100 according to this embodiment may include a panoramic camera 101, a camera 102, a network 103, a server 104, a first terminal device 105, and a second terminal device 106. The network 103 serves as a medium for providing communication links between different devices.

[0040] Panoramic camera 101 and / or camera 102 can be used to capture the video to be processed. Panoramic camera 101 and / or camera 102 can send the video to be processed to server 104 via network 103, so that server 104, in response to a stabilization mode switching command, processes a second video segment of the video to be processed using a second stabilization mode to obtain a first target video segment.

[0041] Alternatively, the video processing provided in this embodiment can also be performed by the first terminal device 105 and / or the second terminal device 106. That is, in response to a stabilization mode switching command, the first terminal device 105 and / or the second terminal device 106 process the second video segment of the video to be processed using a second stabilization mode to obtain the first target video segment.

[0042] It should be understood that the number of panoramic cameras, cameras, networks, servers, first terminal devices, and second terminal devices in Figure 1 is merely illustrative. Depending on the implementation requirements, any number of panoramic cameras, cameras, networks, servers, first terminal devices, and second terminal devices can be used.

[0043] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0044] The system architecture for applying video processing methods provided in this disclosure has been described above. The video processing process of this disclosure will be further explained below using Figure 2 as an example.

[0045] Figure 2 schematically illustrates a flowchart of a video processing method according to an embodiment of the present disclosure.

[0046] As shown in Figure 2, the video processing method 200 may include step S210.

[0047] In step S210, in response to the stabilization mode switching command, the second video segment of the video to be processed is processed using the second stabilization mode to obtain the first target video segment. The video to be processed also includes the first video segment, which has the stabilization effect corresponding to the first stabilization mode, and the first stabilization mode is different from the second stabilization mode.

[0048] A stabilization mode switching command is a signal or instruction used to change the stabilization method for specific segments of a video being processed. In one example, a user can manually trigger the stabilization mode switching command via a button, touchscreen gesture, or voice command. In another example, the server can automatically trigger the stabilization mode switching command by performing scene analysis on the video being processed.

[0049] For example, users can tap the image stabilization mode switch button on the terminal device to trigger the image stabilization mode switch command; alternatively, users can trigger the image stabilization mode switch command by performing a corresponding gesture swipe operation on the touch screen; alternatively, users can trigger the image stabilization mode switch command by sending "I want to switch image stabilization modes" to the voice processing component of the terminal device.

[0050] For example, the server can perform motion detection on the video to be processed, and automatically trigger a command to switch the image stabilization mode when it detects video segments with motion. Alternatively, the server can perform light change detection on the video to be processed, and automatically trigger a command to switch the image stabilization mode when it detects video segments with light changes.

[0051] The video to be processed may include at least a first video segment and a second video segment. The first video segment refers to the portion of the video to be processed that has already been processed using the first stabilization method and has the corresponding stabilization effect. The second video segment refers to the portion of the video to be processed that needs to be processed using the second stabilization method. The second stabilization method refers to a video stabilization processing method different from the first stabilization method. For example, if the first stabilization method is a three-axis locked stabilization method, the second stabilization method can be a stabilization method with at least one axis released; alternatively, if the first stabilization method is a stabilization method with at least one axis released, the second stabilization method can be a three-axis locked stabilization method.

[0052] The method for obtaining the second video clip and the second image stabilization method can be configured according to actual business needs and is not limited here. In one example, the user can manually select the second video clip and the corresponding second image stabilization method through buttons, touchscreen gestures, or voice commands. In another example, the server can automatically determine the second video clip and the corresponding second image stabilization method by performing scene analysis on the video to be processed.

[0053] For example, a user can click the video clip selection button on the terminal device and specify the corresponding time period to manually select the second video clip; alternatively, a user can perform a corresponding gesture swipe operation on the touch screen and specify the corresponding time period to manually select the second video clip; alternatively, a user can manually select the second video clip by sending a message to the voice processing component of the terminal device, "Process the clip from minute 1 to minute 3".

[0054] For example, the server can perform motion detection on the video to be processed. If it detects video segments where the shaking exceeds a first threshold, it identifies these segments as second video segments and automatically determines a second stabilization method appropriate to the shaking. In one example, the second stabilization method appropriate to the shaking can be determined based on a first mapping. This first mapping can be a pre-configured table or set representing the correspondence between candidate shaking conditions and candidate stabilization methods. In this case, the shaking condition can be matched with each candidate shaking condition in the first mapping, and the candidate stabilization method corresponding to the candidate shaking condition that matches the shaking condition can be determined as the second stabilization method.

[0055] For example, the server can detect changes in lighting in the video being processed. If a video segment shows a lighting change exceeding a second threshold, that segment is identified as the second video segment, and a second image stabilization method adapted to the lighting change is automatically determined. In one example, the second image stabilization method adapted to the lighting change can be determined based on a second mapping. This second mapping can be a pre-configured table or set representing the correspondence between candidate lighting changes and candidate image stabilization methods. In this case, the lighting changes can be matched with each candidate lighting change in the second mapping, and the candidate image stabilization method corresponding to the candidate lighting change that matches the lighting change can be determined as the second image stabilization method.

[0056] After determining the second video segment and the second image stabilization method, the second video segment in the video to be processed can be processed using the second image stabilization method to obtain the first target video segment. The first target video segment refers to the second video segment processed by the second image stabilization method. The first target video segment has the image stabilization effect corresponding to the second image stabilization method.

[0057] It should be noted that the video processing method 200 provided in this disclosure can be applied to multiple fields such as tourism, live sports events, concerts, education, architecture, film and television production, game development, and outdoor sports, without limitation. Correspondingly, the video to be processed can be at least one of the following: tourism videos, live sports event videos, concert videos, educational videos, architectural demonstration videos, film and television production videos, game development videos, outdoor sports videos, etc., without limitation.

[0058] For example, in travel videos shot in the tourism sector, a first image stabilization method can be applied to a first video clip shot at a fixed location, focusing on close-ups of the scenery, while a second image stabilization method can be applied to a second video clip shot from a moving vehicle on a highway. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be a stabilization method with at least one axis released. By applying different image stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, travel memories can be made clearer and more vivid, thereby enhancing the viewer's viewing experience.

[0059] For example, in live sports broadcasts, a first stabilization method can be applied to the first video clip showing the formation, the stands, or the stadium environment from a full-view perspective. A second stabilization method can be applied to the second video clip following a football player's breakthrough. The first stabilization method could be a three-axis locked stabilization method, while the second stabilization method could be a stabilization method with at least one axis released. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, the footage in high-speed motion scenes becomes stable and clear, improving the quality of the live broadcast and the viewer's experience.

[0060] For example, in concert video recording, a first image stabilization method can be applied to the first video clip shot from a fixed camera position, depicting the main stage light show. A second image stabilization method can be applied to the second video clip, which tracks the singer's movement in follow spot mode. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be at least one axis unlocked. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can be more immersed in the musical atmosphere, improving the quality of the live concert broadcast and the overall viewing experience.

[0061] For example, in educational videos within a teaching setting, a first stabilization method can be applied to the first video segment of the teacher writing on the blackboard, while a second stabilization method can be applied to the second video segment showing the pouring of liquids in a chemical experiment. The first stabilization method could be a three-axis locked stabilization method, while the second stabilization method could be a stabilization method with at least one axis released. By applying different stabilization methods to the relatively static first video segment and the relatively dynamic second video segment, the quality of the educational videos is improved, making the teaching content clearer and easier to understand, thus helping students learn and comprehend knowledge more effectively.

[0062] For example, in architectural demonstration videos, a first image stabilization method can be applied to the first video clip showing the building's exterior and overall structure, while a second image stabilization method can be applied to the second video clip showing interior details or the construction process. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be at least one axis deactivated. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can gain a clearer understanding of the building's details and overall structure, ensuring a more stable and clear presentation of these aspects.

[0063] For example, in film and television production scenarios, a first image stabilization method can be applied to the first video clip of a dialogue scene shot on a track, while a second image stabilization method can be applied to the second video clip of a handheld action scene. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be a stabilization method with at least one axis released. By applying different image stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can be more immersed in the storyline, improving the picture quality and viewing experience of the film or television work.

[0064] For example, in game development videos, a first stabilization method can be applied to the first video clip playing CG (Computer Graphics, opening animation), and a second stabilization method can be applied to the second video clip of the first-person shooter combat scene. The first stabilization method could be a three-axis locked stabilization method, and the second stabilization method could be a stabilization method with at least one axis released. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, players can more clearly understand the game's content and features, improving the stability and viewing experience of the game videos.

[0065] For example, in skiing videos set in outdoor sports scenarios, a first stabilization method can be applied to the first video clip of low-speed skiing practice, while a second stabilization method can be applied to the second video clip of downhill skiing or obstacle jumping. The first stabilization method could be a three-axis locked stabilization method, while the second stabilization method could be a stabilization method with at least one axis released. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can clearly see the skiing movements and snow scenery, improving the stability and viewing experience of the skiing videos.

[0066] For example, in cycling videos set in outdoor sports scenarios, a first image stabilization method can be applied to the first video clip of cruising on flat urban roads, while a second image stabilization method can be applied to the second video clip of riding on bumpy mountain trails. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be a stabilization method with at least one axis released. By applying different image stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can better experience the joy of cycling, enhancing the video's viewing experience.

[0067] For example, in outdoor sports scenes like mountain climbing videos, a first image stabilization method can be applied to the first video clip shot from a fixed point on the summit, while a second image stabilization method can be applied to the second video clip shot handheld during the climb. The first image stabilization method could be a three-axis locked stabilization method, while the second image stabilization method could be at least one axis unlocked. By applying different stabilization methods to the relatively static first video clip and the relatively dynamic second video clip, viewers can better appreciate the natural scenery, improving the stability and clarity of the mountain climbing video.

[0068] According to embodiments of this disclosure, by responding to a stabilization mode switching command, different stabilization modes can be applied to different video segments in the video being processed. This not only effectively reduces video shake but also avoids the monotony caused by using the same stabilization mode for the entire video, improving the flexibility and adaptability of video stabilization processing, thereby enhancing the stability and viewing experience of the entire video. Furthermore, the above method can also meet users' personalized needs for video stabilization in different scenarios, enhancing the video shooting and viewing experience.

[0069] The above provides a preliminary description of the video processing process provided in this disclosure. The following will use Figures 3A and 3B as examples to further explain the process of obtaining the second video segment.

[0070] Figure 3A schematically illustrates an example of a second video segment acquisition process according to one embodiment of the present disclosure; Figure 3B schematically illustrates an example of a second video segment acquisition process according to another embodiment of the present disclosure.

[0071] In embodiments of this disclosure, the second video segment can be obtained by responding to a time period selected from the video to be processed. The second video segment refers to the video content corresponding to a specific continuous or non-continuous time period determined from the video to be processed according to preset rules or user selection. A time period refers to one or more continuous intervals on the video playback timeline, used to identify a portion of the video to be processed. For example, if the duration of the video to be processed is 10 minutes and the time period is from the 3rd minute to the 5th minute, then the video content corresponding to the 3rd minute to the 5th minute can be identified.

[0072] In one example, the time period can be manually selected by the user. For example, the time period can be determined by at least one of the following methods: inputting the start and end time points through interactive operations of a GUI (Graphical User Interface), marking the start and end time points through voice commands, mapping the start and end time points through eye-tracking gaze areas, determining the start and end time points through dialogue context, etc.

[0073] For example, interactive operations may include at least one of the following: dragging a timeline with a mouse or touchscreen, selecting start and end time points, typing start and end time points in a text box or search bar, selecting start and end time points from a drop-down menu or tab, etc. Dialogue context refers to the information and environmental background provided by previous interactions with the user, which helps the server understand the current communication content and intent. Voice commands refer to sound signals captured and processed by the terminal device.

[0074] In another example, the time period can also be automatically determined according to preset rules. For instance, the time period can be determined by analyzing the content of the video to be processed, automatically recommending at least one candidate time period for the user to choose from, and using the time period selected by the user from the at least one candidate time period as the time period.

[0075] For example, motion detection can be performed on the video to be processed to identify time periods with significant motion changes, and these periods can be used as the second video segment. Alternatively, lighting change detection can be performed on the video to be processed to identify time periods with significant lighting changes, and these periods can be used as the second video segment. Alternatively, audio detection can be performed on the video to be processed to identify time periods with sudden increases in audio waveforms, and these periods can be used as the second video segment. Alternatively, content analysis can be performed on the video to be processed to obtain keywords or tags for different segments, and based on these keywords or tags, time periods of user interest can be selected and used as the second video segment.

[0076] It should be noted that there can be one or more time periods. If there is only one target time period, the video content corresponding to that time period in the video to be processed can be directly used as the second video segment. For example, if the video to be processed is 5 minutes long and the user selects the 30th to 90th second as the time period, then the video content corresponding to the 30th to 90th second can be used as the second video segment. If there are multiple time periods, then the video content corresponding to each of the multiple time periods in the video to be processed can be used as multiple second video segments. For example, if the video to be processed is 5 minutes long and the user selects the 10th-20th second and the 50th-60th second as time periods, then the video content corresponding to the 10th-20th second can be used as the first second video segment, and the video content corresponding to the 50th-60th second can be used as the second second video segment.

[0077] As shown in Figure 3A, in 300A, the process of determining the second video segment is schematically illustrated by means of buttons and touch screen gestures.

[0078] For example, users can click the time period selection button on the front-end interface to trigger the time period selection operation. Based on this, users can select time point 1 and time point 2 on the progress bar at the bottom of the front-end interface to determine the time period between time point 1 and time point 2 as the time period selected from the video to be processed, and determine the video content corresponding to the time period between time point 1 and time point 2 as the second video segment.

[0079] As shown in Figure 3B, in 300B, the process of determining the second video segment is schematically illustrated by using buttons and input boxes.

[0080] For example, users can click the time period selection button on the front-end interface to trigger the time period selection operation. Based on this, users can enter the start time as time point 3 and the end time as time point 4 on the displayed time period selection interface to determine the time period between time point 3 and time point 4 as the time period selected from the video to be processed, and determine the video content corresponding to the time period between time point 3 and time point 4 as the second video segment.

[0081] It should be noted that the front-end interface used for time period selection can be presented in various ways. For example, it can be presented through buttons and touch screen gestures as shown in Figure 3A, or through buttons and input boxes as shown in Figure 3B, or other presentation methods, which are not limited here.

[0082] According to embodiments of this disclosure, by obtaining a second video segment in response to a time period selected from the video to be processed, whether the user selects it manually or the server automatically identifies and selects it, the required video segment can be determined quickly and accurately, thereby meeting the user's personalized needs for video segments in different scenarios and improving the targeting and efficiency of video processing.

[0083] The above provides a preliminary description of the process for obtaining the second video segment provided in this disclosure. The following will use Figures 4A and 4B as examples to illustrate the image stabilization effect of each video segment in the video to be processed.

[0084] Figure 4A schematically illustrates an example of the image stabilization effect of each video segment in a video to be processed according to an embodiment of the present disclosure; Figure 4B schematically illustrates an example of the image stabilization effect of each video segment in a video to be processed according to another embodiment of the present disclosure.

[0085] In the embodiments of this disclosure, the second video segment may have the stabilization effect corresponding to the initial stabilization method, and the first target video segment may have the stabilization effect corresponding to the second stabilization method. The initial stabilization method refers to the stabilization processing method applied to the second video segment before the stabilization method is switched. In one example, the first stabilization method and the initial stabilization method may be the same or different.

[0086] As shown in Figure 4A, in 400A, an example of the image stabilization effect of each video segment in the video to be processed is illustrated when the first image stabilization method is the same as the initial image stabilization method.

[0087] For example, the video to be processed 401 may include a first video segment 402 and a second video segment 403. Both the first video segment 402 and the second video segment 403 are processed using the initial stabilization method. Therefore, the first video segment 402 can have the stabilization effect corresponding to the initial stabilization method, and the second video segment 403 can also have the stabilization effect corresponding to the initial stabilization method. In response to a stabilization method switching command, the second video segment 403 of the video to be processed 401 can be processed using the second stabilization method to obtain a first target video segment 404. Since the first target video segment 404 is processed using the second stabilization method, it has the stabilization effect corresponding to the second stabilization method.

[0088] As shown in Figure 4B, in 400B, an example of the image stabilization effect of each video segment in the video to be processed is illustrated when the first image stabilization method is different from the initial image stabilization method.

[0089] For example, the video to be processed 401 may include a first video segment 405, a third video segment 406, and a second video segment 407. The first video segment 405 is processed using a first stabilization method, and therefore has the stabilization effect corresponding to the first stabilization method. The third video segment 406 and the second video segment 407 are both processed using an initial stabilization method, and therefore the third video segment 406 and the second video segment 407 each have the stabilization effect corresponding to the initial stabilization method. In response to a stabilization method switching command, the second video segment 407 of the video to be processed 401 can be processed using a second stabilization method to obtain a first target video segment 408. Since the first target video segment 408 is processed using the second stabilization method, it has the stabilization effect corresponding to the second stabilization method.

[0090] According to embodiments of this disclosure, since the second video segment has the stabilization effect corresponding to the initial stabilization method, the first video segment has the stabilization effect corresponding to the first stabilization method, and the first target video segment has the stabilization effect corresponding to the second stabilization method, and the first stabilization method is different from the second stabilization method, and the first stabilization method and the initial stabilization method can be the same or different, the stabilization method can be dynamically adjusted according to the specific shaking situation and processing requirements of the video segment, thereby achieving flexible stabilization processing of the video segment. This ensures that the video obtains the best stabilization effect in different scenarios, which helps to improve the viewing experience of the video.

[0091] In the embodiments of this disclosure, in response to the image stabilization mode switching command, the second video segment of the video to be processed can be processed with the second image stabilization mode to obtain the first target video segment. The above process will be described by way of example below with reference to FIG5A and FIG5B.

[0092] Figure 5A schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method according to an embodiment of the present disclosure to obtain a first target video segment.

[0093] As shown in Figure 5A, in 500A, in response to the stabilization mode switching instruction 501, the second video segment 503 of the video to be processed is processed with the second stabilization mode to obtain the first target video segment 504. This may include: in response to the stabilization mode switching instruction 501, determining a processing strategy 502 corresponding to the second stabilization mode, the processing strategy 502 being used to compensate for the shake of the second stabilization mode relative to the initial stabilization mode; and processing the second video segment 503 based on the processing strategy 502 to obtain the first target video segment 504.

[0094] Processing strategy 502 refers to a method developed to achieve switching between different image stabilization methods, guiding how to perform shake compensation on a second video segment to switch the image stabilization method applied to the second video segment from the initial image stabilization method to the second image stabilization method. For example, if the initial image stabilization method is a three-axis locked image stabilization method and the second image stabilization method is an image stabilization method with at least one axis released, then processing strategy 502 may include an image stabilization camera movement adjustment operation based on at least one released axis corresponding to the second image stabilization method.

[0095] In one example, the processing strategy 502 can be automatically determined according to preset rules. For example, the processing strategy 502 for compensating for shake in the second stabilization method relative to the initial stabilization method can be determined based on a third mapping, an initial stabilization method, and a second stabilization method. The third mapping can include the correspondence between first candidate stabilization methods and candidate stabilization setting parameters. After determining the initial stabilization method and the second stabilization method, the initial stabilization method can be matched with multiple first candidate stabilization methods in the third mapping, and the candidate stabilization setting parameter corresponding to the first candidate stabilization method that matches the initial stabilization method can be determined as the first target stabilization setting parameter; the second stabilization method can be matched with multiple first candidate stabilization methods in the third mapping, and the candidate stabilization setting parameter corresponding to the first candidate stabilization method that matches the second stabilization method can be determined as the second target stabilization setting parameter; thus, the processing strategy 502 is determined based on the first target stabilization setting parameter and the second target stabilization setting parameter.

[0096] Alternatively, a processing strategy 502 for compensating for shake in the second stabilization method relative to the initial stabilization method can be determined based on the fourth mapping, the initial stabilization method, and the second stabilization method. The fourth mapping may include the correspondence between the second candidate stabilization method, the third candidate stabilization method, and candidate stabilization conversion parameters. After determining the initial stabilization method and the second stabilization method, they can be matched with multiple second and third candidate stabilization methods in the fourth mapping. The candidate stabilization conversion parameters corresponding to the second candidate stabilization method that matches the initial stabilization method and the third candidate stabilization method that matches the second stabilization method are determined as the target stabilization conversion parameters. Thus, the processing strategy 502 is determined based on the target stabilization conversion parameters.

[0097] According to embodiments of this disclosure, a processing strategy is determined by responding to a stabilization mode switching command, and the second video segment is processed based on the strategy. This allows for dynamic adjustment of the stabilization mode and processing strategy according to the specific shaking condition and processing requirements of the video segment, thus achieving flexible stabilization processing of the video segment.

[0098] Figure 5B schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method according to another embodiment of the present disclosure to obtain a first target video segment.

[0099] As shown in Figure 5B, in 500B, in response to the image stabilization mode switching command 505, the processing strategy corresponding to the second image stabilization mode is determined, which may include: obtaining the relative attitude information sequence 508 of the second image stabilization mode relative to the initial image stabilization mode; generating the corresponding first image stabilization camera movement data 510 based on the relative attitude information sequence 508, and determining the processing strategy based on the first image stabilization camera movement data 510.

[0100] The relative attitude information sequence 508 refers to a time-series data set describing the differences in motion compensation between two image stabilization methods. This time-series data set may include information on the relative direction and relative position changes between the second image stabilization method and the initial image stabilization method, arranged in a time sequence. The relative attitude information sequence 508 may, for example, be acquired using sensors on a video capture device. In one example, the relative attitude information sequence 508 may include differences in X-axis translation, Y-axis translation, Z-axis translation, Roll axis rotation, Pitch axis rotation, and Yaw axis rotation.

[0101] For example, if the initial stabilization method is a three-axis locked stabilization method and the second stabilization method is a stabilization method with at least one axis released, then the relative attitude information sequence 508 may include attitude change data of the video acquisition device at different time points, such as rotation angle, displacement, etc. These attitude change data can reflect the relative change of the attitude of the video acquisition device during the process of switching from the three-axis locked stabilization method to the stabilization method with at least one axis released.

[0102] In the embodiments of this disclosure, obtaining the relative attitude information sequence 508 of the second stabilization method relative to the first stabilization method may include: obtaining the first attitude information sequence 506 corresponding to the initial stabilization method and the second attitude information sequence 507 corresponding to the second stabilization method; and determining the relative attitude information sequence 508 based on the first attitude information sequence 506 and the second attitude information sequence 507.

[0103] The first attitude information sequence 506 refers to the time sequence set of motion data of the video acquisition device when the initial image stabilization mode is applied. That is, under the initial image stabilization mode, the attitude information of the video acquisition device at various time points during the shooting process, which may include, for example, position and orientation. The second attitude information sequence 507 refers to the time sequence set of motion data of the video acquisition device when the second image stabilization mode is applied. That is, under the second image stabilization mode, the attitude information of the video acquisition device at various time points during the shooting process, which may include, for example, position and orientation.

[0104] After obtaining the first attitude information sequence 506 and the second attitude information sequence 507, a relative attitude information sequence 508 can be determined based on the first attitude information sequence 506 and the second attitude information sequence 507. For example, the attitude information difference for each time point can be determined by comparing the attitude information in the first attitude information sequence 506 and the attitude information in the second attitude information sequence 507 for the same time point, and the attitude information difference for each time point can be determined as the relative attitude information sequence 508.

[0105] According to embodiments of this disclosure, by obtaining the attitude information sequences of the initial stabilization method and the second stabilization method respectively, and determining the relative attitude information sequence based on these sequences, the stabilization method and processing strategy can be dynamically adjusted according to the actual attitude changes during video shooting, which helps to enhance the video viewing experience.

[0106] The first image stabilization camera movement data 510 refers to the set of image stabilization motion parameters generated based on the relative attitude information sequence 508, which is used to guide video image stabilization processing. For example, the first image stabilization camera movement data 510 may include the movement direction, movement position, rotation angle, etc. of the lens of the video capture device required to convert the initial image stabilization mode to the second image stabilization mode. The movement direction may include, for example, the horizontal direction and the vertical direction, and the movement position may include a first compensation amount corresponding to the horizontal direction and a second compensation amount corresponding to the vertical direction.

[0107] In the embodiments of this disclosure, generating the corresponding first image-stabilized camera movement data 510 based on the relative attitude information sequence 508 may include: smoothing the relative attitude information sequence 508 to obtain a smoothed relative attitude information sequence 509; and generating the first image-stabilized camera movement data 510 based on the smoothed relative attitude information sequence 509.

[0108] Smoothing refers to filtering or optimizing the relative attitude information sequence 508 to reduce noise and abrupt changes in the data, making the data smoother and more continuous. The specific method of smoothing can be configured according to actual business needs and is not limited here. For example, smoothing the relative attitude information sequence 508 can reduce jitter in attitude changes, making them more natural. The smoothed relative attitude information sequence 509 refers to the smoothed relative attitude information sequence, which is smoother and more continuous with less jitter than the original relative attitude information sequence 508.

[0109] For example, smoothing methods can include at least one of the following: Kalman filter-based smoothing methods, low-pass filter-based smoothing methods, moving average-based smoothing methods, and artificial intelligence-based smoothing methods. Artificial intelligence-based smoothing methods refer to using a first neural network to process the relative pose information sequence 508 to predict trajectory smoothness, and then using a second neural network to repair outliers based on the smoothness. For example, the first neural network can be an LSTM (Long Short-Term Memory) network, and the second neural network can be a GAN (Generative Adversarial Network).

[0110] The specific smoothing method chosen can be configured according to actual business needs and is not limited here. In one example, the smoothing method can be automatically selected based on the scene of the second video segment or based on the relative jitter type.

[0111] For example, taking the selection of a smoothing method based on the scene of the second video segment as an example: if the second video segment is a dynamic scene, a smoothing method with strong smoothing ability for dynamic scenes can be selected; if the second video segment is a static scene, a smoothing method with strong smoothing ability for static scenes can be selected. Alternatively, taking the selection of a smoothing method based on the type of relative jitter as an example: if the relative jitter type is translation, a smoothing method with strong smoothing ability for relative translation jitter can be selected; if the relative jitter type is rotation, a smoothing method with strong smoothing ability for relative rotation jitter can be selected; if the relative jitter type is composite jitter, a smoothing method with strong smoothing ability for both relative translation jitter and relative rotation jitter can be selected simultaneously, or a smoothing method with strong smoothing ability for both relative translation jitter and relative rotation jitter can be selected overall.

[0112] According to embodiments of this disclosure, by acquiring a relative pose information sequence and smoothing it to generate a smooth relative pose information sequence, abrupt changes in video jitter can be effectively reduced, making the video footage smoother and more natural. Based on this, by generating first anti-shake camera movement data based on the smooth relative pose information sequence, precise control of video stabilization is achieved, improving the accuracy and reliability of the stabilization process.

[0113] The first image stabilization camera movement data 510 refers to the specific data generated based on a smoothed relative attitude information sequence, used to guide video image stabilization processing. It includes the camera movement parameters required to achieve stabilization, such as the lens's movement direction, speed, and rotation angle. For example, it includes data such as the lens's movement compensation in the horizontal, vertical, and longitudinal directions, as well as rotation compensation angles, calculated based on the smoothed relative attitude information sequence. This data guides the lens movement to achieve better image stabilization.

[0114] In the embodiments of this disclosure, processing the second video segment 511 to obtain the first target video segment 512 based on the processing strategy may include: performing shake compensation on the second video segment 511 based on the first anti-shake camera movement data 510 to obtain the first target video segment 512 with the anti-shake effect of the second anti-shake method.

[0115] Shake compensation refers to the process of compensating for and correcting the shake in the second video segment 511 where the second stabilization method differs from the initial stabilization method, using specific algorithms and processing methods. This allows the stabilization method used in the second video segment 511 to be switched from the initial stabilization method to the second stabilization method. The first target video segment 512 refers to the video segment obtained after shake compensation processing, which exhibits the stabilization effect of the second stabilization method.

[0116] The specific method of jitter compensation can be configured according to actual business needs and is not limited here. For example, jitter compensation methods can include at least one of hardware-based jitter compensation methods and software-based jitter compensation methods. Hardware-based jitter compensation methods refer to using hardware devices such as three-axis gimbal stabilizers to adjust the posture of the video acquisition device in real time by setting different anti-shake modes to achieve three-axis locking or at least one-axis release. Software-based jitter compensation methods refer to using video editing software or anti-shake algorithms to perform post-processing on the video to be processed by setting different anti-shake modes to achieve three-axis locking or at least one-axis release.

[0117] According to embodiments of this disclosure, by performing shake compensation on the second video segment based on the first image stabilization camera movement data, it is possible to compensate for shake that differs between the second image stabilization method and the initial image stabilization method. This results in the first target video segment having the image stabilization effect corresponding to the second image stabilization method, while other first video segments in the video to be processed have the image stabilization effect corresponding to the first image stabilization method. Since the first image stabilization method is different from the second image stabilization method, different video segments of the entire video can use different image stabilization methods, which helps to improve the accuracy and reliability of image stabilization processing, thereby enhancing the video viewing experience.

[0118] According to embodiments of this disclosure, by acquiring a sequence of relative posture information and generating first anti-shake camera movement data based on this to determine a processing strategy, the anti-shake method and processing strategy can be dynamically adjusted according to the actual posture changes during video shooting, thereby improving the stability and viewing experience of the video and achieving precise control over video anti-shake.

[0119] In the embodiments of this disclosure, the video to be processed can be a panoramic video to be processed, and the second video segment can be a panoramic video segment to be processed; the initial stabilization method can refer to a three-axis locked stabilization method, and the second stabilization method can refer to a stabilization method with at least one axis released.

[0120] Panoramic videos can be captured using any shooting device, such as panoramic cameras, gimbal cameras, drones, mobile phones, and smart glasses. A panoramic camera can include, for example, a fisheye lens. In one example, a panoramic camera can be equipped with at least two fisheye lenses deployed opposite each other. Images are captured through these fisheye lenses and then stitched together to obtain a 360° panoramic image or video. Users can control the display viewing angle to view objects in the image from left to right and top to bottom, achieving a 360° surround view of a three-dimensional space.

[0121] The three axes can be, for example, the Roll axis, the Pitch axis, and the Yaw axis. The Roll axis is the axis that tilts the camera left and right; the motor controlling the Roll axis adjusts the camera's left and right tilt to prevent left and right camera shake. The Pitch axis is the axis that moves the camera up and down; the motor controlling the Pitch axis adjusts the camera's forward and backward tilt to prevent up and down camera shake. The Yaw axis is the axis that rotates the camera left and right; the motor controlling the Yaw axis adjusts the camera's left and right rotation to prevent camera shake caused by left and right camera rotation.

[0122] Three-axis image stabilization refers to image stabilization that locks and stabilizes on three axes. In other words, the initial image stabilization method stabilizes shake along the axes of left-right tilt, up-down movement of the shooting angle, and left-right rotation of the shooting angle.

[0123] A stabilization method with at least one axis released refers to a method where at least one of the three axes is not locked, allowing it to move freely within a certain range. For example, if the Yaw axis is released, the second stabilization method allows shake along the left-right rotation axis of the shooting angle, while stabilizing shake along the left-right tilt axis and the up-down movement axis of the shooting angle. Alternatively, if the Pitch axis is released, the second stabilization method allows shake along the up-down movement axis of the shooting angle, while stabilizing shake along the left-right tilt axis and the left-right rotation axis of the shooting angle. Alternatively, if the Roll axis is released, the second stabilization method allows shake along the left-right tilt axis, while stabilizing shake along the up-down movement axis of the shooting angle and the left-right rotation axis of the shooting angle.

[0124] According to embodiments of this disclosure, by providing a three-axis locked stabilization method and a stabilization method with at least one axis released, the stabilization method of different video segments can be flexibly adjusted according to the specific needs and scene characteristics of video shooting, providing a more flexible choice for video stabilization.

[0125] The following, in conjunction with Figures 6A and 6B, illustrates the process of processing the second video segment of the video to be processed with the second stabilization method in response to a stabilization method switching command, in the case where the second video segment is a panoramic video segment to be processed, the initial stabilization method is a three-axis locked stabilization method, and the second stabilization method is a stabilization method with at least one axis released, to obtain the first target video segment.

[0126] Figure 6A schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method to obtain a first target video segment, according to yet another embodiment of the present disclosure.

[0127] As shown in Figure 6A, in 600A, in response to the image stabilization mode switching instruction 601, the second video segment of the video to be processed is processed with the second image stabilization mode to obtain the first target video segment 604. This may include: obtaining the second image stabilization camera movement data 602 corresponding to the second image stabilization mode; and performing image stabilization camera movement processing on the first playback angle of the panoramic video segment 603 to be processed based on the second image stabilization camera movement data 602 to obtain the first playback angle with the effect of the second image stabilization mode.

[0128] The second image stabilization camera movement data 602 refers to specific data generated according to the second image stabilization method, used to guide video stabilization processing. It may include camera movement parameters required to achieve stabilization, such as the lens's movement direction, speed, and rotation angle. In one example, the camera movement can be changed from at least one-axis loose stabilization to a three-axis locked stabilization method. Therefore, if you want to switch the initial stabilization method used in the second video segment 603 to the second stabilization method, you can determine the second image stabilization camera movement data 602 based on the direction of the locked camera movement of the second stabilization method compared to the initial stabilization method. For example, in the at least one-axis loose stabilization method, the second image stabilization camera movement data 602 may include the movement compensation amount and rotation compensation angle of the video capture device's lens in the loose axis. The movement compensation amount can be a fixed orientation, such as towards the position (0, 0, 0), which remains unchanged throughout the process.

[0129] The second image stabilization camera movement data 602 can be automatically determined according to preset rules. For example, the second image stabilization camera movement data 602 corresponding to the second image stabilization method can be determined based on the fifth mapping and the second image stabilization method. The fifth mapping can include a fourth candidate image stabilization method and candidate image stabilization camera movement data of the fourth candidate image stabilization method relative to the three-axis locked image stabilization method. After determining the second image stabilization method, multiple fourth candidate image stabilization methods can be matched according to the second image stabilization method and the fifth mapping, and the candidate image stabilization camera movement data corresponding to the fourth candidate image stabilization method that matches the second image stabilization method can be determined as the second image stabilization camera movement data 602.

[0130] The panoramic video clip 603 to be processed can be a video frame from a first playback perspective, which refers to the local field of view of the panoramic video currently being viewed by the user. The first playback perspective can have first perspective parameters, such as at least one of a first yaw parameter, a first pitch parameter, a first roll parameter, and a first field of view (Fov) parameter. The second image stabilization camera movement data 602 can include second perspective parameters, such as at least one of a second yaw parameter, a second pitch parameter, a second roll parameter, and a second field of view (Fov) parameter. Correspondingly, image stabilization camera movement processing can refer to at least one of adjusting the first yaw parameter based on the second yaw parameter, adjusting the first pitch parameter based on the second pitch parameter, adjusting the first roll parameter based on the second roll parameter, and adjusting the first field of view parameter based on the second field of view parameter.

[0131] According to embodiments of this disclosure, by acquiring second stabilization camera movement data corresponding to the second stabilization method, and performing stabilization camera movement processing on the first playback angle of the panoramic video clip based on this data, the stabilization method and processing strategy can be flexibly adjusted according to the specific needs and scene characteristics of video shooting, thereby achieving precise control of video stabilization.

[0132] Figure 6B schematically illustrates an example of a process in which a second video segment of a video to be processed is processed using a second stabilization method to obtain a first target video segment, according to another embodiment of the present disclosure.

[0133] As shown in Figure 6B, in 600B, obtaining the second image stabilization movement data 608 corresponding to the second image stabilization mode may include: in response to the image stabilization mode switching command 605, obtaining the original attitude information sequence 606 corresponding to the panoramic video segment 609 to be processed, wherein the original attitude information sequence 606 is obtained by the camera detecting the camera's attitude during the process of shooting the panoramic video segment 609 to be processed; and generating the second image stabilization movement data 608 corresponding to the second image stabilization mode based on the original attitude information sequence 606.

[0134] The original attitude information sequence 606 can be obtained by detecting the camera's attitude. Attitude detection refers to acquiring the camera's position and orientation in three-dimensional space in real time through a preset detection method. The original attitude information sequence 606 can include multiple view parameters of the camera at each time point, such as yaw, pitch, roll, and field of view (Fov) parameters.

[0135] In one example, the camera's attitude can be detected in real time during the capture of the panoramic video clip 609 to be processed using hardware sensors, resulting in a raw attitude information sequence 606 characterizing the camera's attitude changes. For example, the hardware sensors may include at least one of a gyroscope, accelerometer, magnetometer, and GPS.

[0136] In another example, a visual algorithm can be used to detect the camera's pose in real time while the camera is capturing the panoramic video segment 609 to be processed, thereby obtaining a raw pose information sequence 606 that characterizes the camera's pose changes. For example, the visual algorithm may include at least one of SLAM (Simultaneous Localization and Mapping) and optical flow analysis.

[0137] According to embodiments of this disclosure, by acquiring the original posture information sequence and generating second anti-shake camera movement data based on it, the anti-shake method and processing strategy can be dynamically adjusted according to the actual posture changes during video shooting, thereby achieving precise anti-shake processing of panoramic video clips.

[0138] In the embodiments of this disclosure, after obtaining the original attitude information sequence 606, second image stabilization camera movement data 608 corresponding to the second image stabilization method can be generated based on the original attitude information sequence 606. This can include: obtaining at least one release axis corresponding to the second image stabilization method, determining the original axial attitude information sequence 607 corresponding to the at least one release axis, and obtaining the second image stabilization camera movement data 608 corresponding to the second image stabilization method based on the original axial attitude information sequence 607 corresponding to the at least one release axis. After obtaining the second image stabilization camera movement data 608 corresponding to the second image stabilization method, the second video segment 609 can be processed based on the second image stabilization camera movement data 608 to obtain the first target video segment 610.

[0139] A release axis refers to an axis that is not locked in the second image stabilization method and is allowed to move freely within a certain range. After determining the second image stabilization method, at least one release axis corresponding to the second image stabilization method can be determined. For each release axis, a sequence of original axial attitude information 607 corresponding to that release axis is determined. The sequence of original axial attitude information 607 refers to the sequence of attitude change information for each axis obtained by attitude detection of the camera's release axes during the process of the camera shooting panoramic video.

[0140] In one example, if the release axis corresponding to the second image stabilization method is the Yaw axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the left-right rotation axis of the shooting angle; alternatively, if the release axis corresponding to the second image stabilization method is the Pitch axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the up-down movement axis of the shooting angle; alternatively, if the release axis corresponding to the second image stabilization method is the Roll axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the left-right tilt axis.

[0141] In another example, if the release axis corresponding to the second image stabilization method is the Yaw axis and the Pitch axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the left-right rotation axis of the shooting angle and an attitude change information sequence along the up-down movement axis of the shooting angle; if the release axis corresponding to the second image stabilization method is the Yaw axis and the Roll axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the left-right rotation axis of the shooting angle and an attitude change information sequence along the left-right tilt axis; if the release axis corresponding to the second image stabilization method is the Pitch axis and the Roll axis, then the original axial attitude information sequence 607 can be an attitude change information sequence along the up-down movement axis of the shooting angle and an attitude change information sequence along the left-right tilt axis.

[0142] In another example, if the release axis corresponding to the second image stabilization method is the Yaw axis, Pitch axis, and Roll axis, then the original axial attitude information sequence 607 can be an attitude change information sequence on the axis of left and right rotation along the shooting angle, an attitude change information sequence on the axis of up and down movement along the shooting angle, and an attitude change information sequence on the axis of left and right tilt.

[0143] According to embodiments of this disclosure, by acquiring the release axis and its original axial attitude information sequence corresponding to the second stabilization method, and generating second stabilization camera movement data based on this, the stabilization method and processing strategy can be dynamically adjusted according to the actual attitude changes during video shooting, thereby achieving precise control of video stabilization.

[0144] In embodiments of this disclosure, the video processing method 200 may further include: in response to an editing operation on a first target video segment, processing the first target video segment based on the editing operation to obtain a second target video segment.

[0145] Editing operations refer to further processing operations performed by the user on the first target video clip with the stabilization effect corresponding to the second stabilization method. The second target video clip refers to the final video clip obtained after the editing operations. In one example, the default viewpoint displayed when the user has not performed any other editing operations can be understood as the bottom viewpoint. For example, the bottom viewpoint can be the target stabilization camera movement data of the second stabilization method relative to the initial stabilization method. The camera movement data generated based on the user's editing operations can be understood as the top viewpoint. For the first target video clip, if the bottom viewpoint and the top viewpoint overlap, it is necessary to determine the interaction logic between the two and process the first target video clip based on the interaction logic to obtain the final second target video clip.

[0146] In one example, the editing operation may include at least one of a first editing type operation and a second editing type operation. The first editing type operation may be an editing operation dependent on the initial stabilization method, meaning that the first editing type operation is related to the initial stabilization method of the second video segment, i.e., the first editing type operation is performed based on a bottom-view angle. The second editing type operation may be an editing operation independent of the initial stabilization method, meaning that the type operation is not related to the initial stabilization method of the second video segment, i.e., the second editing type operation is not performed based on a bottom-view angle.

[0147] In the embodiments of this disclosure, after obtaining the first target video segment, a first editing type operation can be performed on the first target video segment. The first editing type operation may include: tracking operation, dot-mapping operation, and forward direction selection operation. The following will use Figures 7A to 7C to exemplarily illustrate the process of processing the first target video segment based on different first editing type operations to obtain the second target video segment.

[0148] Figure 7A schematically illustrates an example of a process of processing a first target video segment based on a tracking operation to obtain a second target video segment according to an embodiment of the present disclosure.

[0149] As shown in Figure 7A, Figure 700A schematically illustrates the process of processing the first target video segment based on the tracking operation to obtain the second target video segment when the first editing type operation is a tracking operation.

[0150] When shooting moving scenes, the relative positions of the camera and the subject can easily change, potentially causing the subject's position to shift within the video clip. Tracking operations can be used to automatically track the object, keeping it fixed in the frame. Tracking operations refer to the process of tracking and recording specific objects or features in a video during editing. For example, tracking a moving object in a video and recording its trajectory can be used for later addition of special effects or adjustments to the image.

[0151] Taking the 700A as an example, you can trigger the tracking operation by long-pressing the screen of the first target video clip and manually selecting the object you want to track, thereby locking the object to be tracked in the screen and fixing the object in the screen.

[0152] For the first target video segment with the stabilization effect corresponding to the second stabilization method, since stabilization processing affects the motion trajectory and position of objects in the video, the position of keyframes, and the direction of object movement, the tracking operation depends on the initial stabilization method. Because the upper-level viewpoint corresponding to the tracking operation does not change due to other factors, if the lower-level viewpoint and the upper-level viewpoint corresponding to the tracking operation coincide, the upper-level viewpoint corresponding to the tracking operation will cover the lower-level viewpoint; that is, the upper-level viewpoint corresponding to the tracking operation will be used as the viewpoint of the second target video segment.

[0153] Therefore, when the editing operation is a tracking operation, the tracking operation can be directly superimposed on a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying a second target video clip. In this case, the second target video clip is a video clip that has been processed based on the tracking operation and has a stabilization effect corresponding to the initial stabilization method.

[0154] It should be noted that there are multiple ways to trigger the tracking operation. For example, it can be triggered by long-pressing the screen and manually selecting the target object, as shown in Figure 7A. Alternatively, a tracking button can be set on the front-end interface, and clicking the tracking button will intelligently identify the object to be tracked and trigger the tracking operation. Other triggering methods are also possible, which are not limited here.

[0155] Figure 7B schematically illustrates an example of a process for processing a first target video segment based on a dot-mapping operation to obtain a second target video segment according to an embodiment of the present disclosure.

[0156] As shown in Figure 7B, in 700B, the process of processing the first target video segment based on the dot operation to obtain the second target video segment is illustrated when the first editing type operation is a dot operation.

[0157] For panoramic videos, users can control the display perspective to view objects in the image from left to right and top to bottom, achieving a 360° surround view of a three-dimensional space. Keyframes can be added to fix the controlled display perspective for later camera movement. Keyframe marking refers to the operation of marking keyframes or specific moments in a video during editing. For example, marking moments or locations in the video that require special effects processing allows for precise editing later.

[0158] Taking 700B as an example, you can drag the screen of the first target video clip and adjust it to the desired viewpoint. Then, you can set a keyframe at the corresponding video frame to fix that viewpoint. Furthermore, by dragging the timeline and repeatedly dragging the screen, adjusting the viewpoint, and setting keyframes, you can control the viewpoint at different points in time, thus smoothly transitioning the video clip from the viewpoint set in the first keyframe to the viewpoint set in the next keyframe.

[0159] For the first target video segment with the stabilization effect corresponding to the second stabilization method, since stabilization processing affects the motion trajectory and position of objects in the video, the position of keyframes, and the direction of object movement, the dot-matrix operation depends on the initial stabilization method. Because the upper-level viewpoint corresponding to the dot-matrix operation does not change due to other factors, if the lower-level viewpoint and the upper-level viewpoint corresponding to the dot-matrix operation coincide, the upper-level viewpoint corresponding to the dot-matrix operation will cover the lower-level viewpoint; that is, the upper-level viewpoint corresponding to the dot-matrix operation will be used as the viewpoint of the second target video segment.

[0160] Therefore, when the editing operation is a dot-matrix operation, the dot-matrix operation can be directly superimposed onto a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying the second target video clip. In this case, the second target video clip is a video clip that has been processed based on the dot-matrix operation and has a stabilization effect corresponding to the initial stabilization method.

[0161] It should be noted that there are multiple ways to trigger the point-marking operation. For example, it can be triggered by dragging the screen, adjusting the view, and marking a keyframe, as shown in Figure 7B. Alternatively, a point-marking button can be set on the front-end interface, and clicking the button can intelligently identify at least one keyframe and the view corresponding to each keyframe to trigger the point-marking operation. Other triggering methods are also possible, which are not limited here.

[0162] Figure 7C schematically illustrates an example of a process of processing a first target video segment to obtain a second target video segment based on a forward direction selection operation according to an embodiment of the present disclosure.

[0163] For panoramic videos, users can select a direction of travel to view the scene corresponding to that direction. Selecting a direction of travel refers to choosing the direction of travel for an object or viewpoint within the video during editing. For example, when shooting a moving scene, selecting the camera's direction of travel allows for subsequent camera movement or special effects processing.

[0164] Taking the 700C as an example, users can use a swipe gesture on the front-end interface to move from the first position to the second position, and the relative position change between the first and second positions indicates the way forward.

[0165] For the first target video segment with the stabilization effect corresponding to the second stabilization method, since stabilization processing affects the motion trajectory and position of objects in the video, the position of keyframes, and the direction of object movement, the forward direction selection operation depends on the initial stabilization method. Because the upper-level viewpoint corresponding to the forward direction selection operation does not change due to other factors, if the bottom viewpoint coincides with the upper-level viewpoint corresponding to the forward direction selection operation, the upper-level viewpoint corresponding to the forward direction selection operation will cover the bottom viewpoint; that is, the upper-level viewpoint corresponding to the forward direction selection operation will be used as the viewpoint of the second target video segment.

[0166] Therefore, when the editing operation is a forward direction selection operation, the forward direction selection operation can be directly superimposed on the second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying the second target video clip. In this case, the second target video clip is a video clip that has been processed based on the forward direction selection operation and has a stabilization effect corresponding to the initial stabilization method.

[0167] It should be noted that there are multiple ways to trigger the forward direction selection operation. For example, it can be triggered by a swipe gesture on the front-end interface as shown in Figure 7C. Alternatively, a forward direction selection button can be set on the front-end interface, and clicking the forward direction selection button will intelligently identify the expected forward direction to trigger the forward direction selection operation. Other triggering methods are also possible, which are not limited here.

[0168] Therefore, when the editing operation is a first editing type operation, the first editing type operation can be directly superimposed on a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying a second target video clip. In this case, the second target video clip is a video clip that has been processed based on the first editing type operation and has a stabilization effect corresponding to the initial stabilization method.

[0169] In the embodiments of this disclosure, after obtaining the first target video segment, a second editing type operation can be performed on the first target video segment. The second editing type operation may include a free viewpoint adjustment operation. The process of processing the first target video segment based on the second editing type operation to obtain the second target video segment will be exemplarily described below with reference to FIG7D.

[0170] Figure 7D schematically illustrates an example of a process of processing a first target video segment to obtain a second target video segment based on a free-viewpoint adjustment operation according to an embodiment of the present disclosure.

[0171] As shown in Figure 7D, in 700D, the process of processing the first target video segment based on the free viewpoint adjustment operation to obtain the second target video segment is illustrated when the second editing type operation is a free viewpoint adjustment operation.

[0172] For panoramic videos, users can freely adjust the display perspective to view objects in the image from left to right and from top to bottom, achieving a 360° surround view of a three-dimensional space. Free perspective adjustment refers to the ability to adjust the video's perspective during editing, without being limited by the initial stabilization method. For example, users can rotate, zoom, or pan the video frame to obtain the desired viewing angle.

[0173] Taking Figure 7D as an example, you can drag the screen of the first target video clip and adjust it to the desired viewpoint to view the display of the video clip from a specific perspective.

[0174] For the first target video segment with the stabilization effect corresponding to the second stabilization method, since the free angle adjustment operation mainly focuses on the user's personalized adjustment of the angle, the free angle adjustment operation does not depend on the initial stabilization method. If the bottom angle and the upper angle corresponding to the free angle adjustment operation coincide, the upper angle corresponding to the free angle adjustment operation will be superimposed on the bottom angle. That is, the superimposed angle of the upper angle and the bottom angle corresponding to the free angle adjustment operation is used as the angle of the second target video segment.

[0175] Therefore, when the editing operation is a free-viewpoint adjustment operation, the free-viewpoint adjustment operation is directly superimposed on the first target video segment with the second stabilization method to obtain the second target video segment. This second target video segment is a video segment that is processed based on the free-viewpoint adjustment operation on the first target video segment and has a stabilization effect corresponding to the second stabilization method.

[0176] In one example, the video to be processed may further include a first intermediate video segment following the video segment to be processed. This first intermediate video segment is a video processed using the initial stabilization method. For the first intermediate video segment, when the editing operation is a free-viewpoint adjustment operation, the free-viewpoint adjustment operation can be superimposed on the first intermediate video segment to obtain a third target video segment. In this case, the third target video segment is a video segment based on the free-viewpoint adjustment operation that processes the first intermediate video segment and has a stabilization effect corresponding to the second stabilization method.

[0177] In the embodiments of this disclosure, after obtaining the first target video segment, a first editing type operation and a second editing type operation can be performed on the first target video segment simultaneously. The process of processing the first target video segment based on the first editing type operation and the second editing type operation to obtain the second target video segment will be illustrated below with reference to Figures 7E to 7G.

[0178] Figure 7E schematically illustrates an example of a process in which a first target video segment is processed simultaneously based on a tracking operation and a free-viewpoint adjustment operation according to an embodiment of the present disclosure to obtain a second target video segment.

[0179] As shown in Figure 7E, in 700E, the process of processing the first target video segment simultaneously based on the tracking operation and the free-viewpoint adjustment operation to obtain the second target video segment is illustrated.

[0180] As shown above, if the bottom viewpoint coincides with both the upper viewpoint corresponding to the tracking operation and the upper viewpoint corresponding to the free viewpoint adjustment operation, then the upper viewpoint corresponding to the tracking operation will cover the bottom viewpoint, and the upper viewpoint corresponding to the free viewpoint adjustment operation will be superimposed on the upper viewpoint. That is, the upper viewpoint corresponding to the tracking operation is used as the first viewpoint of the second target video segment, and the upper viewpoint corresponding to the free viewpoint adjustment operation is superimposed on the first viewpoint to obtain the second viewpoint of the second target video segment.

[0181] Therefore, when the editing operation is a tracking operation and a free-viewpoint adjustment operation, these operations can be directly superimposed onto a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying the second target video clip. In this case, the second target video clip is a video clip that has been processed based on the tracking operation and the free-viewpoint adjustment operation, and has a stabilization effect corresponding to the initial stabilization method.

[0182] In one example, the video to be processed may further include a second intermediate video segment following the initial video segment, which is a video processed using the initial stabilization method. For the second intermediate video segment, when the editing operation is a tracking operation and a free-viewpoint adjustment operation, the tracking operation can be superimposed on the second intermediate video segment to obtain a fourth target video segment. In this case, the fourth target video segment is a video segment based on the tracking operation that processes the second video segment and has a stabilization effect corresponding to the initial stabilization method.

[0183] Figure 7F schematically illustrates an example of a process for processing a first target video segment to obtain a second target video segment according to an embodiment of the present disclosure based on dot-mapping operations and free-viewpoint adjustment operations.

[0184] As shown in Figure 7F, Figure 700F schematically illustrates the process of simultaneously processing the first target video segment based on the dot operation and the free-viewpoint adjustment operation to obtain the second target video segment when the editing operation is a dot operation and a free-viewpoint adjustment operation.

[0185] As shown above, if the bottom view coincides with both the upper view corresponding to the dot operation and the upper view corresponding to the free view adjustment operation, then the upper view corresponding to the dot operation will cover the bottom view, and the upper view corresponding to the free view adjustment operation will be superimposed on the upper view. That is, the upper view corresponding to the dot operation is used as the first view of the second target video segment, and the upper view corresponding to the free view adjustment operation is superimposed on the first view to obtain the second view of the second target video segment.

[0186] Therefore, when the editing operation is a dot-matrix operation and a free-viewpoint adjustment operation, these operations can be directly superimposed onto a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying the second target video clip. In this case, the second target video clip is a video clip that has been processed based on the dot-matrix operation and the free-viewpoint adjustment operation, and has a stabilization effect corresponding to the initial stabilization method.

[0187] In one example, the video to be processed may further include a second intermediate video segment following the initial video segment, which is a video processed using the initial stabilization method. For the second intermediate video segment, when the editing operation is a dot-matrix operation and a free-viewpoint adjustment operation, the dot-matrix operation can be superimposed on the second intermediate video segment to obtain a fourth target video segment. In this case, the fourth target video segment is a video segment based on the dot-matrix operation that processes the second intermediate video segment and has a stabilization effect corresponding to the initial stabilization method.

[0188] Figure 7G schematically illustrates an example of a process of processing a first target video segment to obtain a second target video segment based on a forward direction selection operation and a free viewpoint adjustment operation according to an embodiment of the present disclosure.

[0189] As shown in Figure 7G, Figure 700G schematically illustrates the process of simultaneously processing the first target video segment based on the forward direction selection operation and the free viewpoint adjustment operation to obtain the second target video segment when the editing operation is the forward direction selection operation and the free viewpoint adjustment operation.

[0190] As shown above, if the bottom view coincides with both the upper view corresponding to the forward direction selection operation and the upper view corresponding to the free view adjustment operation, then the upper view corresponding to the forward direction selection operation will cover the bottom view, and the upper view corresponding to the free view adjustment operation will be superimposed on the upper view. That is, the upper view corresponding to the forward direction selection operation is used as the first view of the second target video segment, and the upper view corresponding to the free view adjustment operation is superimposed on the first view to obtain the second view of the second target video segment.

[0191] Therefore, when the editing operation involves selecting the forward direction and adjusting the free angle, these operations can be directly superimposed onto a second video clip that has a stabilization effect corresponding to the initial stabilization method, resulting in and displaying the second target video clip. In this case, the second target video clip is a video clip that has been processed based on the forward direction selection and free angle adjustment operations and has a stabilization effect corresponding to the initial stabilization method.

[0192] In one example, the video to be processed may further include a second intermediate video segment following the initial video segment, which is processed using the initial stabilization method. For the second intermediate video segment, when the editing operation is a forward direction selection operation and a free-viewpoint adjustment operation, the forward direction selection operation can be overlaid on the second intermediate video segment to obtain a fourth target video segment. In this case, the fourth target video segment is a video segment that has processed the second intermediate video segment based on the forward direction selection operation and has a stabilization effect corresponding to the initial stabilization method.

[0193] According to embodiments of this disclosure, by distinguishing between a first editing type operation and a second editing type operation, and superimposing them onto different video segments according to the operation type, the personalized needs of users for video content in different scenarios can be met, realizing flexible customization and optimization of video content, and effectively improving the efficiency and quality of video editing.

[0194] According to embodiments of this disclosure, by distinguishing between editing operations that depend on the initial stabilization method and those that do not, and by overlaying them onto different video segments according to the operation type, the personalized needs of users for video content in different scenarios can be met, enabling flexible customization and optimization of video content, and providing strong support for video creation and application.

[0195] The above describes the process of processing the second video segment of the video to be processed with the second stabilization method in response to the stabilization mode switching command to obtain the first target video segment, and the process of processing the first target video segment based on the editing operation in response to the first target video segment to obtain the second target video segment. The following will use Figure 8 to give an overall description of the video processing process provided by the embodiments of this disclosure.

[0196] Figure 8 schematically illustrates an example diagram of a video processing procedure according to an embodiment of the present disclosure.

[0197] As shown in Figure 8, taking the video to be processed 801, which includes video segment 1 from time 0 to time point t1, video segment 2 from time point t1 to time point t2, video segment 3 from time point t2 to time point t3, video segment 4 from time point t3 to time point t4, and video segment 5 from time point t4 to time point t5, as an example, the video processing process is explained.

[0198] The video to be processed, 801, can be a video segment that has applied the initial stabilization method. That is, at the beginning of video processing, video segments 1 to 5 all have stabilization effects corresponding to the initial stabilization method. Since the stabilization method of video segment 1 from time 0 to time t1 was not switched, video segment 1 from time 0 to time t1 still has the stabilization effect corresponding to the initial stabilization method.

[0199] In response to a stabilization mode switching command for switching the initial stabilization mode to a first stabilization mode, video segment 2 from time point t1 to time point t2 can be processed using the first stabilization mode to obtain video segment 2' with a stabilization effect corresponding to the first stabilization mode. In response to a stabilization mode switching command for switching the initial stabilization mode to a second stabilization mode, video segment 3 from time point t2 to time point t3, video segment 4 from time point t3 to time point t4, and video segment 5 from time point t4 to time point t5 can be processed using the second stabilization mode to obtain video segment 3', video segment 4', and video segment 5' with a stabilization effect corresponding to the second stabilization mode.

[0200] In response to the first edit type operation on video segments 3' and 4', and in response to the second edit type operation on video segments 4' and 5', for video segment 3', since only the first edit type operation was performed, the first edit type operation can be directly superimposed on video segment 3 with the stabilization effect corresponding to the initial stabilization method, resulting in video segment 3'", which is equivalent to canceling the corresponding second stabilization operation. For video segment 4', since both the first and second edit type operations were performed, the first and second edit type operations can be directly superimposed on video segment 4 with the stabilization effect corresponding to the initial stabilization method, resulting in video segment 4'", which is equivalent to canceling the corresponding second stabilization operation in this time segment. For video segment 5', since only the second edit type operation was performed, the second edit type operation can be directly superimposed on video segment 5' with the second stabilization method, resulting in video segment 5'.

[0201] The above are merely exemplary embodiments, but are not limited thereto. Other video processing methods known in the art may also be included, as long as they can improve the flexibility and adaptability of video stabilization processing.

[0202] Figure 9 schematically illustrates a block diagram of a video processing apparatus according to an embodiment of the present disclosure. The electronic device shown in Figure 9 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0203] As shown in FIG9, the computer video processing device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 909 into a random access memory (RAM) 903. For example, the memory 902 may store instructions executable by the processor 901. The instructions are executed by the processor 901 to cause the processor 901 to process a second video segment of the video to be processed in response to a stabilization mode switching instruction, using a second stabilization mode to obtain a first target video segment; wherein the video to be processed also includes a first video segment, the first video segment having a stabilization effect corresponding to a first stabilization mode, the first stabilization mode being different from the second stabilization mode.

[0204] Processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 901 may also include onboard memory for caching purposes. Processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this disclosure.

[0205] In some embodiments, the second video segment has a stabilization effect corresponding to the initial stabilization method, and the first target video segment has a stabilization effect corresponding to the second stabilization method.

[0206] In some embodiments, when the processor 901 processes a second video segment of the video to be processed using a second stabilization method in response to a stabilization mode switching instruction to obtain a first target video segment, it is configured to: determine a processing strategy corresponding to the second stabilization method in response to the stabilization mode switching instruction, wherein the processing strategy is used to perform shake compensation on the second stabilization method relative to the initial stabilization method; and process the second video segment based on the processing strategy to obtain the first target video segment.

[0207] In some embodiments, when the processor 901 determines the processing strategy corresponding to the second image stabilization method, it is configured to: obtain a relative attitude information sequence of the second image stabilization method relative to the initial image stabilization method; generate corresponding first image stabilization camera movement data based on the relative attitude information sequence; and determine the processing strategy based on the first image stabilization camera movement data.

[0208] In some embodiments, when the processor 901 obtains the relative attitude information sequence of the second stabilization method relative to the initial stabilization method, it is used to: obtain the first attitude information sequence corresponding to the initial stabilization method and the second attitude information sequence corresponding to the second stabilization method; and determine the relative attitude information sequence based on the first attitude information sequence and the second attitude information sequence.

[0209] In some embodiments, when the processor 901 generates corresponding first image-stabilized camera movement data based on the relative attitude information sequence, it is used to: smooth the relative attitude information sequence to obtain a smoothed relative attitude information sequence; and generate the first image-stabilized camera movement data based on the smoothed relative attitude information sequence.

[0210] In some embodiments, when the processor 901 processes the second video segment based on a processing strategy to obtain the first target video segment, it is used to: perform shake compensation on the second video segment based on the first anti-shake camera movement data to obtain the first target video segment with the anti-shake effect of the second anti-shake method.

[0211] In some embodiments, the second video segment is a panoramic video segment to be processed; the initial stabilization method refers to a three-axis locked stabilization method, and the second stabilization method refers to a stabilization method with at least one axis released.

[0212] In some embodiments, when the processor 901 responds to the image stabilization mode switching instruction and processes the second video segment of the video to be processed with the second image stabilization mode to obtain the first target video segment, it is used to: obtain the second image stabilization camera movement data corresponding to the second image stabilization mode; and perform image stabilization camera movement processing on the first playback angle of the panoramic video segment to be processed based on the second image stabilization camera movement data to obtain the first playback angle with the effect of the second image stabilization mode.

[0213] In some embodiments, when the processor 901 acquires the second image stabilization camera movement data corresponding to the second image stabilization method, it is used to: acquire the original attitude information sequence corresponding to the panoramic video segment to be processed, wherein the original attitude information sequence is obtained by the camera detecting the camera's attitude during the process of shooting the panoramic video segment to be processed; and generate the second image stabilization camera movement data corresponding to the second image stabilization method based on the original attitude information sequence.

[0214] In some embodiments, when the processor 901 generates second image stabilization camera movement data corresponding to the second image stabilization method based on the original attitude information sequence, it is used to: obtain at least one release axis corresponding to the second image stabilization method, determine the original axial attitude information sequence corresponding to the at least one release axis, and obtain the second image stabilization camera movement data corresponding to the second image stabilization method based on the original axial attitude information sequence corresponding to the at least one release axis.

[0215] In some embodiments, the processor 901 is further configured to: in response to an editing operation on a first target video segment, process the first target video segment based on the editing operation to obtain a second target video segment.

[0216] In some embodiments, the editing operation includes at least one of a first editing type operation and a second editing type operation, wherein the first editing type operation is an editing operation dependent on an initial image stabilization method, and the second editing type operation is an editing operation independent of an initial image stabilization method; when the processor 901 processes the first target video segment based on the editing operation to obtain a second target video segment, it is configured to: when the editing operation is a first editing type operation, superimpose the editing operation onto the second video segment to obtain the second target video segment; when the editing operation is a second editing type operation, superimpose the editing operation onto the first target video segment to obtain the second target video segment; and when the editing operation is both a first editing type operation and a second editing type operation, superimpose the editing operation onto the second video segment to obtain the second target video segment.

[0217] In some embodiments, the first type of editing operation includes: tracking operation, dot operation, and forward direction selection operation; the second type of editing operation includes: free viewpoint adjustment operation.

[0218] In some embodiments, the second video segment is obtained by the processor 901 in response to a time period selected from the video to be processed.

[0219] RAM 903 stores various programs and data required for the operation of the panoramic video playback device 900. The processor 901, ROM 902, and RAM 903 are interconnected via bus 904.

[0220] According to embodiments of this disclosure, the panoramic video playback device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The panoramic video playback device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0221] Figure 10 schematically illustrates a video processing system according to an embodiment of the present disclosure.

[0222] As shown in Figure 10, the video processing system 1000 may include the video acquisition device 1010 and the video processing device 1020 in any of the embodiments described above.

[0223] The video acquisition device 1010 is used to acquire video to be processed. In some embodiments, the panoramic video acquisition device 1010 may include at least one of a panoramic camera, a gimbal camera, a drone, a mobile phone, and smart glasses.

[0224] The video processing device 1020 is used to process a second video segment of the video to be processed in response to a stabilization mode switching command, using a second stabilization mode to obtain a first target video segment; wherein the video to be processed also includes a first video segment, the first video segment has a stabilization effect corresponding to a first stabilization mode, and the first stabilization mode is different from the second stabilization mode.

[0225] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on a panoramic video playback device, the program code is used to enable the panoramic video playback device to implement the video processing methods provided in the embodiments of this disclosure.

[0226] When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, modules, units, etc., described above can be implemented by computer program modules.

[0227] According to embodiments of this disclosure, program code for executing computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages.

[0228] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. It should also be noted that in some alternative implementations, the functions indicated in the boxes may occur in a different order than those shown in the drawings.

[0229] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method of video processing, the method comprising: The method includes: In response to the stabilization mode switching command, the second video segment of the video to be processed is processed with the second stabilization mode to obtain the first target video segment; The video to be processed also includes a first video segment, which has a stabilization effect corresponding to a first stabilization method, and the first stabilization method is different from the second stabilization method.

2. The method of claim 1, wherein, The second video segment has the stabilization effect corresponding to the initial stabilization method, and the first target video segment has the stabilization effect corresponding to the second stabilization method.

3. The method of claim 2, wherein, The step of processing the second video segment of the video to be processed using the second stabilization method in response to the stabilization mode switching command to obtain the first target video segment includes: In response to the image stabilization mode switching command, a processing strategy corresponding to the second image stabilization mode is determined, and the processing strategy is used to perform shake compensation on the second image stabilization mode relative to the initial image stabilization mode; Based on the processing strategy, the second video segment is processed to obtain the first target video segment.

4. The method of claim 3, wherein, The process strategy for determining the second image stabilization method includes: Obtain the relative attitude information sequence of the second stabilization method relative to the initial stabilization method; Based on the relative attitude information sequence, corresponding first image stabilization camera movement data is generated, and the processing strategy is determined based on the first image stabilization camera movement data.

5. The method of claim 4, wherein, The step of obtaining the relative attitude information sequence of the second stabilization method relative to the first stabilization method includes: Obtain the first attitude information sequence corresponding to the initial anti-shake method and the second attitude information sequence corresponding to the second anti-shake method; The relative attitude information sequence is determined based on the first attitude information sequence and the second attitude information sequence.

6. The method of claim 4, wherein, The step of generating corresponding first image-stabilized camera movement data based on the relative attitude information sequence includes: The relative attitude information sequence is smoothed to obtain a smoothed relative attitude information sequence; The first image-stabilized camera movement data is generated based on the smooth relative attitude information sequence.

7. The method of claim 4, wherein, The step of processing the second video segment based on the processing strategy to obtain the first target video segment includes: Based on the first image stabilization camera movement data, the second video segment is subjected to shake compensation to obtain the first target video segment with the image stabilization effect of the second image stabilization method.

8. The method of claim 1, wherein, The second video segment is a panoramic video segment to be processed; the initial stabilization method refers to a three-axis locked stabilization method, and the second stabilization method refers to a stabilization method with at least one axis released.

9. The method of claim 8, wherein, The step of processing the second video segment of the video to be processed using the second stabilization method in response to the stabilization mode switching command to obtain the first target video segment includes: Obtain the second image stabilization camera movement data corresponding to the second image stabilization method; Based on the second anti-shake camera movement data, the first playback angle of the panoramic video clip to be processed is subjected to anti-shake camera movement processing to obtain a first playback angle with the effect of the second anti-shake method.

10. The method of claim 9, wherein, The step of obtaining the second image stabilization movement data corresponding to the second image stabilization method includes: Obtain the original pose information sequence corresponding to the panoramic video segment to be processed. The original pose information sequence is obtained by the camera detecting the camera's pose during the process of shooting the panoramic video segment to be processed. Based on the original attitude information sequence, second image stabilization camera movement data corresponding to the second image stabilization method is generated.

11. The method of claim 10, wherein, The step of generating second image stabilization movement data corresponding to the second image stabilization method based on the original attitude information sequence includes: Obtain at least one release axis corresponding to the second anti-shake method, and determine the original axial attitude information sequence corresponding to at least one release axis; Based on the original axial attitude information sequence corresponding to the at least one release axis, the second image stabilization camera movement data corresponding to the second image stabilization method is obtained.

12. The method of claim 8, wherein, The method further includes: In response to an editing operation on the first target video segment, the first target video segment is processed based on the editing operation to obtain a second target video segment.

13. The method of claim 12, wherein, The editing operation includes at least one of a first editing type operation and a second editing type operation, wherein the first editing type operation is an editing operation that depends on the initial image stabilization method, and the second editing type operation is an editing operation that does not depend on the initial image stabilization method; The step of processing the first target video segment based on the editing operation to obtain the second target video segment includes: When the editing operation is the first editing type operation, the editing operation is superimposed on the second video segment to obtain the second target video segment; When the editing operation is the second type of editing operation, the editing operation is superimposed on the first target video segment to obtain the second target video segment; When the editing operation is the first editing type operation and the second editing type operation, the editing operation is superimposed on the second video segment to obtain the second target video segment.

14. The method of claim 13, wherein, The first type of editing operation includes: tracking operation, dot operation, and forward direction selection operation; the second type of editing operation includes: free view adjustment operation.

15. The method of claim 1, wherein, The second video clip was obtained in the following way: In response to the time period selected from the video to be processed, the second video segment is obtained.

16. A video processing apparatus, comprising: One or more processors, working individually or together; as well as A memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to process a second video segment of the video to be processed in a second stabilization mode in response to a stabilization mode switching instruction, thereby obtaining a first target video segment; wherein the video to be processed further includes a first video segment, the first video segment having a stabilization effect corresponding to a first stabilization mode, the first stabilization mode being different from the second stabilization mode.

17. The apparatus of claim 16, wherein, The second video segment has the stabilization effect corresponding to the initial stabilization method, and the first target video segment has the stabilization effect corresponding to the second stabilization method.

18. The apparatus of claim 17, wherein, In response to the stabilization mode switching command, when the processor processes the second video segment of the video to be processed using the second stabilization mode to obtain the first target video segment, it is used to: In response to the image stabilization mode switching command, a processing strategy corresponding to the second image stabilization mode is determined, and the processing strategy is used to perform shake compensation on the second image stabilization mode relative to the initial image stabilization mode; Based on the processing strategy, the second video segment is processed to obtain the first target video segment.

19. The apparatus of claim 18, wherein, When the processor determines the processing strategy corresponding to the second anti-shake method, it is used to: Obtain the relative attitude information sequence of the second stabilization method relative to the initial stabilization method; Based on the relative attitude information sequence, corresponding first image stabilization camera movement data is generated, and the processing strategy is determined based on the first image stabilization camera movement data.

20. The apparatus of claim 19, wherein, When the processor acquires the relative attitude information sequence of the second stabilization method relative to the initial stabilization method, it is used to: Obtain the first attitude information sequence corresponding to the initial anti-shake method and the second attitude information sequence corresponding to the second anti-shake method; The relative attitude information sequence is determined based on the first attitude information sequence and the second attitude information sequence.

21. The apparatus of claim 19, wherein, When the processor generates corresponding first image stabilization camera movement data based on the relative attitude information sequence, it is used for: The relative attitude information sequence is smoothed to obtain a smoothed relative attitude information sequence; The first image-stabilized camera movement data is generated based on the smooth relative attitude information sequence.

22. The apparatus of claim 19, wherein, When the processor processes the second video segment based on the processing strategy to obtain the first target video segment, it is used to: Based on the first image stabilization camera movement data, the second video segment is subjected to shake compensation to obtain the first target video segment with the image stabilization effect of the second image stabilization method.

23. The device according to claim 17, characterized in that, The second video segment is a panoramic video segment to be processed; the initial stabilization method refers to a three-axis locked stabilization method, and the second stabilization method refers to a stabilization method with at least one axis released.

24. The apparatus of claim 23, wherein, In response to the stabilization mode switching command, when the processor processes the second video segment of the video to be processed using the second stabilization mode to obtain the first target video segment, it is used to: Obtain the second image stabilization camera movement data corresponding to the second image stabilization method; Based on the second anti-shake camera movement data, the first playback angle of the panoramic video clip to be processed is subjected to anti-shake camera movement processing to obtain a first playback angle with the effect of the second anti-shake method.

25. The apparatus of claim 24, wherein, When the processor acquires the second image stabilization movement data corresponding to the second image stabilization mode, it is used for: Obtain the original pose information sequence corresponding to the panoramic video segment to be processed. The original pose information sequence is obtained by the camera detecting the camera's pose during the process of shooting the panoramic video segment to be processed. Based on the original attitude information sequence, second image stabilization camera movement data corresponding to the second image stabilization method is generated.

26. The apparatus of claim 25, wherein, When the processor generates second image stabilization movement data corresponding to the second image stabilization method based on the original attitude information sequence, it is used for: Obtain at least one release axis corresponding to the second anti-shake method, and determine the original axial attitude information sequence corresponding to at least one release axis; Based on the original axial attitude information sequence corresponding to the at least one release axis, the second image stabilization camera movement data corresponding to the second image stabilization method is obtained.

27. The device according to claim 23, characterized in that, The processor is also used for: In response to an editing operation on the first target video segment, the first target video segment is processed based on the editing operation to obtain a second target video segment.

28. The device according to claim 27, characterized in that, The editing operation includes at least one of a first editing type operation and a second editing type operation, wherein the first editing type operation is an editing operation that depends on the initial image stabilization method, and the second editing type operation is an editing operation that does not depend on the initial image stabilization method; When the processor processes the first target video segment based on the editing operation to obtain the second target video segment, it is used to: When the editing operation is the first editing type operation, the editing operation is superimposed on the second video segment to obtain the second target video segment; When the editing operation is the second type of editing operation, the editing operation is superimposed on the first target video segment to obtain the second target video segment; When the editing operation is the first editing type operation and the second editing type operation, the editing operation is superimposed on the second video segment to obtain the second target video segment.

29. The device according to claim 28, characterized in that, The first type of editing operation includes: tracking operation, dot operation, and forward direction selection operation; the second type of editing operation includes: free view adjustment operation.

30. The device according to claim 16, characterized in that, The second video segment was acquired by the processor in the following manner: In response to the time period selected from the video to be processed, the second video segment is obtained.

31. A video processing system, comprising: Video capture equipment, used to capture video to be processed; A video processing device is configured to process a second video segment of the video to be processed using a second stabilization method in response to a stabilization mode switching command, thereby obtaining a first target video segment; wherein the video to be processed further includes a first video segment, the first video segment having a stabilization effect corresponding to a first stabilization method, and the first stabilization method being different from the second stabilization method.

Images