Video processing method and photographing device

Abstract

Provided in the present disclosure are a video processing method and a photographing device. By means of acquiring an original video, a plurality of target video frames can be generated on the basis of the original video, and a target video can be generated on the basis of the plurality of target video frames, wherein the plurality of target video frames have different photographing angles of view around a target object, and a second video playback speed of the target video is slower than a first video playback speed of the original video, such that the target video can provide a bullet-time effect with temporal slowdown and object locking.

Description

Video processing methods and shooting equipment Technical Field

[0001] This disclosure relates to, but is not limited to, a video processing method and a shooting device. Background Technology

[0002] In recent years, with the rapid development of image processing technology and the continuous iteration of shooting equipment, more and more video special effects generation technologies have been widely used in film, advertising, games and many other fields. Bullet time, as a special visual effect, can simultaneously provide changes in both time and space dimensions to capture the action details of a specific target in a high-speed motion state.

[0003] However, using related technologies to process videos and generate videos with bullet time effects has problems such as complex video shooting and processing processes and high costs. It is also difficult to guarantee the locking effect on specific targets in the final product, resulting in a poor user experience. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail in this disclosure. This overview is not intended to limit the scope of the claims.

[0005] This disclosure provides a video processing method and a shooting device.

[0006] The first aspect of this disclosure provides a video processing method, comprising:

[0007] Acquire the original video, which has a first video playback speed;

[0008] Based on the original video, multiple target video frames are generated, and the multiple target video frames have different framing angles around the target object;

[0009] Based on the plurality of target video frames, a target video with a second video playback speed is generated, wherein the second video playback speed is less than the first video playback speed.

[0010] A second aspect of this disclosure provides a shooting device, including a memory and a processor, the memory storing a computer program, which, when executed by the processor, is configured to acquire raw video, the raw video having a first video playback speed;

[0011] Based on the original video, multiple target video frames are generated, and the multiple target video frames have different framing angles around the target object;

[0012] Based on the plurality of target video frames, a target video with a second video playback speed is generated, wherein the second video playback speed is less than the first video playback speed.

[0013] The video processing method and shooting device provided in this disclosure, by acquiring the original video, can generate multiple target video frames from the original video, and generate a target video from the multiple target video frames, thus realizing the automatic generation of the target video. The multiple target video frames have different framing angles around the target object, and the second video playback speed of the target video is lower than the first video playback speed of the original video, enabling the target video to provide a bullet-time effect of time slowing down and target locking. By generating target video frames with framing angles around the target object and slowing down the video playback speed to generate a target video with bullet-time effects, the shooting and processing process of the original video is simplified, the generation cost of the target video is reduced, the final product quality of the target video is guaranteed, and the user experience is improved.

[0014] After reading and understanding the accompanying diagrams and detailed descriptions, the other aspects can be understood. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of these embodiments. In these drawings, similar reference numerals are used to denote similar elements. The drawings described below are some embodiments of the present disclosure, but not all embodiments. Other drawings will be readily available to those skilled in the art based on these drawings without inventive effort.

[0016] Figure 1 is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0017] Figure 2 is a flowchart illustrating a video processing method according to another exemplary embodiment.

[0018] Figure 3 is a flowchart illustrating the generation of target video frames based on the pose information of the shooting device corresponding to each original video frame according to an exemplary embodiment.

[0019] Figure 4 is a flowchart illustrating the processing procedure performed on each original video frame when determining the target direction information corresponding to each original video frame based on the shooting device pose information corresponding to each original video frame, according to an exemplary embodiment.

[0020] Figure 5 is a flowchart illustrating, according to an exemplary embodiment, the generation of target video frames based on each original video frame and its corresponding target direction information.

[0021] Figure 6 is a flowchart illustrating a video processing method according to another exemplary embodiment.

[0022] Figure 7 is a block diagram of a computer device according to an exemplary embodiment.

[0023] Reference numerals: 100, computer equipment; 101, computing unit; 102, ROM; 103, RAM; 104, bus; 105, input / output interface; 106, input unit; 107, output unit; 108, storage unit; 109, communication unit. Detailed Implementation

[0024] The technical solutions of the disclosed embodiments will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be arbitrarily combined with each other.

[0025] In recent years, with the rapid development of image processing technology and the continuous iteration of shooting equipment, more and more video special effects generation technologies have been widely used in many fields such as film, advertising, and games to provide users with rich visual experiences. Bullet time, as a special visual effect, can simultaneously achieve extreme changes in time and space to capture details of specific targets in high-speed motion states such as bullet trajectories and human movements.

[0026] In related technologies, since traditional shooting equipment has difficulty maintaining lock on the target during high-speed movement, it is necessary to construct a shooting scene around the target by using an array of shooting equipment composed of multiple shooting devices, and control the exposure order of each shooting device. It is possible to generate videos with bullet time visual effects by performing post-processing on the exposure images obtained from multiple shooting devices.

[0027] However, using related technologies for video processing results in a large number of shooting devices, leading to insufficient control precision for each device, high prices, and difficult post-processing. This results in complex video shooting and processing processes, excessive costs, and difficulty in guaranteeing the final product quality, leading to a poor user experience.

[0028] This disclosure provides an exemplary embodiment of a video processing method and a shooting device. By acquiring an original video, it can generate multiple target video frames from the original video, and then generate a target video from these multiple target video frames, thus achieving automatic generation of the target video. The multiple target video frames have different framing perspectives around the target object, and the second video playback speed of the target video is slower than the first video playback speed of the original video, enabling the target video to provide a bullet-time effect with time slowdown and target locking. By generating target video frames with framing perspectives around the target object and slowing down video playback speed to generate a target video with bullet-time effects, the process of shooting and processing the original video is simplified, the generation cost of the target video is reduced, the final product quality of the target video is guaranteed, and the user experience is improved.

[0029] In one exemplary embodiment, a video processing method is provided, which is applied to a shooting device. The shooting device may include, for example, an electronic device with shooting function such as a mobile phone or a camera. The shooting device may have one or more lenses. When the shooting device is a camera, it may include cameras with different fields of view such as traditional cameras, wide-angle cameras, and panoramic cameras.

[0030] As shown in Figure 1, the video processing method includes the following steps:

[0031] Step S100: Obtain the original video, which has the first video playback speed.

[0032] In step S100, the shooting device acquires raw video by performing a video shooting function, which serves as the raw material for generating the target video. The shooting device may acquire raw video in response to a user's shooting operation, or in response to certain scene conditions. The raw video has a first video playback speed; exemplarily, the first video playback speed of the raw video is the unprocessed video playback speed, that is, the first video playback speed is consistent with the actual time elapsed. It should be noted that the raw video described in this method refers to video acquired by the shooting device through one of its lenses.

[0033] Step S200: Based on the original video, generate multiple target video frames, each with a different framing perspective around the target object.

[0034] In step S200, a specific target object exists in the original video. The target object can be a person or other object requiring focused shooting. Multiple target video frames can be generated from the original video, so that each target video frame has a different framing perspective around the target object; that is, different target video frames can provide different shooting perspectives relative to the target object. For example, target video frames can be generated using target recognition and tracking technology, or they can be generated based on the orientation of the shooting device relative to the target object in the world coordinate system, so that the framing perspective of each target video frame can revolve around the target object.

[0035] Step S300: Based on multiple target video frames, generate a target video with a second video playback speed, where the second video playback speed is less than the first video playback speed.

[0036] In step S300, a target video with a second video playback speed can be generated based on multiple target video frames using image-to-video synthesis, thereby achieving automatic processing of the original video and automatic generation of the target video. The target video has a second video playback speed, which is slower than the first video playback speed. This means the target video has a time-slowing effect relative to the original video. While the first video playback speed matches the actual time flow, the target video's playback speed is slower than the actual time flow, thus providing a slow-motion visual experience.

[0037] Because the multiple target video frames generated from the target video have different framing angles around the target object, the target video has the effect of locking onto and surrounding the target object. Because the second video of the target video plays at a slower speed than the first video of the original video, the target video has the effect of time slowing down. Thus, the target video can provide a bullet-time visual effect to capture the action details of the target object when the shooting device is in high-speed motion.

[0038] Understandably, compared to setting up multiple shooting devices in related technologies, the above video processing method generates target video frames with a framing perspective around the target object and slows down the video playback speed. It can generate a target video with a bullet time effect based on the original video captured by a single shooting device, which greatly reduces the shooting cost of the original video and simplifies the processing of the original video. It can achieve the addition of the bullet time effect with lower equipment and time costs.

[0039] In this embodiment, by acquiring the original video, multiple target video frames can be generated from the original video, and a target video can be generated from these multiple target video frames, thus achieving automatic generation of the target video. The multiple target video frames have different framing perspectives around the target object, and the second playback speed of the target video is slower than the first playback speed of the original video, enabling the target video to provide a bullet-time effect with time slowdown and target locking. By generating target video frames with framing perspectives around the target object and slowing down video playback speed to generate a target video with bullet-time effects, the process of shooting and processing the original video is simplified, the generation cost of the target video is reduced, the final product quality of the target video is guaranteed, and the user experience is improved.

[0040] In some embodiments, the field of view of the original video is less than 220°.

[0041] The field of view of the original video refers to the field of view of the shooting device used to capture the original video. When the field of view of the original video is less than 220°, the shooting device used to acquire the original video can be any one of a traditional camera, a wide-angle camera, a single fisheye camera, or a dual fisheye camera. After acquiring the original video using the aforementioned shooting device with a field of view less than 220°, the user can generate a target video with a bullet-time effect by generating target video frames with a framing perspective around the target object and slowing down the video playback speed.

[0042] In some embodiments, the field of view of the original video is greater than 70° and less than 120°.

[0043] The field of view of the original video refers to the field of view of the shooting device used to capture the original video. When the field of view of the original video is greater than 70° and less than 120°, the shooting device used to acquire the original video can be a wide-angle camera. After acquiring the original video using a wide-angle camera, the user can generate a target video with a bullet-time effect by generating target video frames with a framing perspective around the target object and slowing down the video playback speed.

[0044] It should be noted that when the field of view of the original video is less than 220° or greater than 70° but less than 120°, i.e. when shooting the original video using a non-panoramic camera, since the shooting device cannot cover a 360° field of view, the user needs to perform specific operations to ensure that the target object is within the field of view of the shooting device. In this way, based on target tracking technology or the relative positional relationship between the shooting device and the target object, target video frames with the field of view surrounding the target object can be generated according to the original video.

[0045] In some embodiments, the original video is a panoramic video with a spherical field of view, and the target video frame is a planar video frame determined based on the panoramic video frame.

[0046] The original video is a panoramic video with a spherical field of view, meaning the camera used to capture it is a panoramic camera with a spherical field of view, capable of covering a 360° field of view. After acquiring the panoramic video using the panoramic camera, the user can determine the corresponding planar video frame from the panoramic video frames to serve as the target video frame. This target video frame will have a planar field of view that differs from the spherical field of view of the panoramic video frames. For example, a hemispherical image closer to the target object can be extracted from the panoramic video frames to serve as the planar video frame, giving the target video frame a framing perspective around the target object within the corresponding planar field of view.

[0047] In some embodiments, during the recording of the original video, the shooting device rotates around the target object, and the shooting device's framing angle is always directed towards the side closest to the target object.

[0048] During the recording of the original video, the camera rotates around the target object, and the camera's framing angle always faces the side closest to the target object. This ensures that the target object is always present in the resulting original video, and that the camera's perspective on the target object changes with the camera's rotation. This guarantees that multiple target video frames generated from the original video will have different framing angles around the target object, providing a basis for generating target videos with a bullet-time effect.

[0049] In this embodiment, during the shooting of the original video, the shooting device rotates around the target object, and the framing angle of the shooting device is always facing the side closer to the target object. This ensures that the target object is always present in the original video and provides a shooting angle that changes the target object. This guarantees that the multiple target video frames generated from the original video can have different framing angles around the target object, providing a basis for the generation of target videos with bullet time effects.

[0050] In some embodiments, the shooting device is a drone with a gimbal camera. The drone hovers rapidly around the target object, and the lens of the gimbal camera always faces the side closest to the target object.

[0051] In some embodiments, during the recording of the original video, the shooting device rotates following an auxiliary line or selfie stick connected to the shooting device to rotate around the target object.

[0052] During the recording of the original video, an auxiliary cable or selfie stick connected to the shooting device can be rotated. The axis around which the rotation is performed becomes the rotation axis of the shooting device, allowing it to rotate around the target object. For example, when the user is the target object, the user can swing the auxiliary cable or selfie stick connected to the shooting device, causing the device to rotate around the user. To ensure the effective generation of target video frames from the original video and target video from the target video frames, the shooting device can be rotated while remaining on a horizontal plane perpendicular to the user's body.

[0053] In this embodiment, during the original video recording process, the shooting device rotates following an auxiliary line or selfie stick connected to it, enabling the shooting device to rotate around the target object. This provides a basis for generating the target video with a bullet-time effect. Users can easily achieve the rotation of the shooting device around the target object and the recording of the original video through simple actions such as rotating the auxiliary line or selfie stick. This simplifies the original video recording process and ensures that the original video can be used to generate the target video frames that meet the requirements, thus improving the user experience.

[0054] In some embodiments, as shown in FIG2, the video processing method further includes the following steps before acquiring the original video:

[0055] Step S410: Obtain the pose information of the shooting device.

[0056] In step S410, before starting to shoot the original video, the pose information of the shooting device is acquired in real time. The pose information of the shooting device is used to characterize the position and attitude of the shooting device at the corresponding moment. For example, the pose information of the shooting device can be the inertial measurement unit data acquired by the shooting device through the inertial measurement unit (IMU).

[0057] Step S420: In response to the shooting device pose information meeting the preset shooting conditions, a video shooting command is generated.

[0058] In step S420, when the pose information of the shooting device meets the preset shooting conditions, it indicates that the pose of the shooting device has undergone a specific change or is in a specific state. At this time, the user intends to shoot the original video and generate the target video, so a video shooting command is generated. The shooting device can use the video shooting command as a trigger command to acquire the original video, so as to start shooting the original video in response to the video shooting command. This allows the user to control the shooting device to start capturing the original video without controlling the shutter of the shooting device, but by controlling the pose of the shooting device.

[0059] For example, when the shooting device pose information indicates that the shooting device is in a state of continuous rotation within a preset time range, it is determined that the shooting device pose information meets the preset shooting conditions, and a video shooting command is generated. The shooting device can automatically trigger the shutter to shoot the original video according to the video shooting command, and automatically perform subsequent video processing to finally obtain a target video with a bullet time effect.

[0060] In this embodiment, by acquiring the pose information of the shooting device and generating a video shooting command when the pose information meets preset shooting conditions, the shooting device can respond to the video shooting command and begin shooting the original video, thus achieving automatic triggering of acquiring the original video. In various specific scenarios, such as when using a selfie stick, the user does not need to control the shutter of the shooting device; they can control the shooting device to shoot and process the original video simply by controlling the pose of the shooting device, further enhancing the user experience.

[0061] In some embodiments, the field of view of the target video is smaller than that of the original video.

[0062] The field of view of the shooting device determines the field of view of the original video. Since the target object is not always centered within the shooting device's field of view when shooting the original video, the original video may contain redundant image elements and lack emphasis on the target object. If the field of view of the target video is kept consistent with the original video, it severely affects the target video's ability to focus on and lock onto the target object. Therefore, configuring the field of view of the target video to be smaller than that of the original video allows for cropping of the video frame by reducing the field of view, thus more clearly highlighting the target object and improving its focus and locking effect.

[0063] In this embodiment, the field of view of the target video is configured to be smaller than that of the original video, so that the target video has a smaller field of view than the original video. This makes it easier to fix the target object in the center of the video screen and increase the screen proportion of the target object. It can more obviously highlight the target object in the video screen and improve the surrounding and locking effect of the target object, thereby further improving the bullet time effect of the target video.

[0064] In some embodiments, the framing range of the target video is smaller than the framing range of the original video.

[0065] The framing range of the shooting device determines the framing range of the original video frame. Since the target object is not always centered within the shooting device's framing range when shooting the original video, the original video may contain redundant image elements and lack emphasis on the target object. If the framing range of the target video frame is kept consistent with the original video, it severely impacts the target video's ability to focus on and lock onto the target object. Therefore, configuring the framing range of the target video frame to be smaller than that of the original video frame allows for cropping of the video frame, more clearly highlighting the target object and thus improving the focus on and locking onto the target object.

[0066] In this embodiment, the framing range corresponding to the video frame of the target video is configured to be smaller than that corresponding to the video frame of the original video. This makes the framing range of the target video smaller than that of the original video, which makes it easier to fix the target object in the center of the video frame and increase the screen proportion of the target object. This can more clearly highlight the target object in the video frame and improve the surrounding and locking effect of the target object, thereby further improving the bullet time effect of the target video.

[0067] In some embodiments, the target object is always kept in the center area of ​​the target video frame.

[0068] To highlight the target object in the target video, when generating the target video frame from the original video, the position of the target object in the original video frame can be determined by, for example, target tracking technology or by the relative positional relationship between the target object and the shooting device. The target video frame is then obtained by cropping the area where the target object is located, so that after the target video is generated from the target video frame, the target object can always remain in the center area of ​​the target video frame.

[0069] The geometric center of this central region is located at the geometric center of the target video frame. The shape and proportion of this central region can be set according to the user's need to highlight the target object. For example, the central region could be a region with the same shape as the target video frame, occupying 80% of the target video frame. Keeping the target object consistently within the central area of ​​the target video frame makes the target object in the video frame stand out more clearly.

[0070] In this embodiment, the target object is always kept in the central area of ​​the target video frame, which can fix the target object in the center of the target video frame, making the target object in the video frame more obviously highlighted, improving the surrounding and locking effect of the target object, and further enhancing the bullet time effect of the target video.

[0071] In some embodiments, the framing perspectives of multiple target video frames change continuously, and the target video has a fixed scene effect; or, there are multiple original videos, and the framing perspectives of multiple target video frames corresponding to each original video change continuously, and the framing perspectives of multiple target video frames corresponding to different original videos revolve around the same target object, and the target video has a scene changing effect.

[0072] When a user shoots a raw video, the shooting device's framing perspective changes continuously within the same scene. The multiple raw video frames are consecutive, each with a continuously changing framing perspective. The target video frames determined from the raw video correspond one-to-one with the raw video frames, also being consecutive and possessing continuously changing framing perspectives. This ensures that multiple target video frames have different framing perspectives around the target object. Since each raw video corresponds to a single scene, the target video generated from the target video frames has a fixed effect within that single scene. For example, a target video generated by a user rotating the shooting device in a snowy scene, in addition to having a bullet-time effect around the user, also has a fixed effect of the snowy scene.

[0073] If a user shoots multiple original videos in various scenes, with the shooting device's framing angle continuously changing in each scene, the multiple original video frames of each original video are consecutive video frames with continuously changing framing angles. The target video frames determined based on each original video correspond one-to-one with the original video frames of each original video, also being consecutive video frames with continuously changing framing angles. Furthermore, the framing angles of the multiple target video frames corresponding to different original videos revolve around the same target object, ensuring that each original video's multiple target video frames have different framing angles around the same target object. Since different original videos correspond to different scenes, the target video frames generated from different original videos correspond to different scenes, resulting in a target video with multiple scene changes. For example, if a user rotates the shooting device to generate two original videos in a snow scene and a seaside scene respectively, the target video generated from these two original videos, in addition to having a bullet-time effect around the user, also has a scene change effect of switching between the snow scene and the seaside scene once or multiple times.

[0074] In this embodiment, the framing perspectives of multiple target video frames corresponding to a single original video continuously change, ensuring that multiple target video frames have different framing perspectives around the target object. This allows the target video to have both a bullet-time effect around the target object and a scene-changing effect. The continuous change of framing perspectives across multiple target video frames corresponding to multiple original videos, and the fact that the framing perspectives of multiple target video frames corresponding to different original videos all revolve around the same target object, ensures that multiple target video frames have different framing perspectives around the same target object. This allows the target video to have both a bullet-time effect around the same target object and a scene-changing effect, generating target videos with richer visual effects, expanding the possibilities of shooting devices, and improving the user experience.

[0075] In some embodiments, generating multiple target video frames based on the original video includes the following steps:

[0076] Target tracking is performed on the original video frames of the original video to obtain each target video frame, so that the framing perspective of each target video frame revolves around the target object.

[0077] When generating multiple target video frames from the original video, for example, the original video frames can be input into a preset target tracking model to perform target tracking on the original video frames, determine the position of the target object in the original video frame, and crop the image of the area where the target object is located to obtain the corresponding target video frame. This allows the framing perspective of the target video frame to surround the target object, providing a basis for generating a target video with target locking effect.

[0078] In this embodiment, target video frames are generated by tracking the original video frames, ensuring that the framing of each target video frame revolves around the target object. This guarantees that the target video generated from the target video frames has a target-locking effect, providing a basis for adding bullet effects. By tracking the original video frames, target image frames can be generated quickly and accurately. This simplifies the original video processing while ensuring target-locking effects in the target video, reducing the time and resources required to generate the target video and improving the user experience.

[0079] In other embodiments, multiple target video frames are generated based on the original video, including the following steps:

[0080] Based on the pose information of the shooting device corresponding to the original video frames, target video frames are generated so that the framing angle of each target video frame revolves around the target object.

[0081] The shooting device pose information is used to characterize the position and attitude of the shooting device at a corresponding moment. When generating multiple target video frames based on the original video, the positional relationship between the shooting device and the target object at the time of shooting of each original video frame can also be determined based on the shooting device pose information corresponding to the original video frame. This allows the position of the target object in the original video frame to be determined, and the area where the target object is located to be cropped to obtain the corresponding target video frame. This ensures that the framing angle of the target video frame can surround the target object, providing a basis for generating a target video with target locking effect.

[0082] In this embodiment, the pose information of the shooting device corresponding to the original video frames is used as the basis for obtaining the target video frames, thus realizing the generation of target video frames. This ensures that the framing angle of each target video frame surrounds the target object, guaranteeing that the target video generated from the target video frames has a target-locking effect, providing a basis for adding bullet effects. Using the pose information of the shooting device corresponding to the original video frames as the basis for obtaining target video frames enables fast and accurate generation of target image frames. While ensuring the target-locking effect of the target video, it simplifies the processing of the original video, reduces the time and resources required to generate the target video, and improves the user experience.

[0083] In some embodiments, the pose information of the imaging device includes inertial measurement unit data acquired by the imaging device.

[0084] The pose information of the shooting device can be obtained by the inertial measurement unit (IMU) of the shooting device through its internal IMU. The IMU, through its built-in accelerometer, gyroscope, magnetometer and other sensors, can detect the acceleration, angular velocity and orientation changes of the shooting device in real time, so as to characterize the position and attitude of the shooting device through the above-mentioned IMU data.

[0085] In this embodiment, the inertial measurement unit (IMU) data acquired by the shooting device is used as the shooting device's pose information. The IMU data can characterize the position and attitude of the shooting device, allowing the determination of the positional relationship between the shooting device and the target object, thereby enabling the determination of the target video frame. IMU data features fast response speed, independence from external signals, and low power consumption, ensuring the efficiency and accuracy of acquiring the target video frame based on the shooting device's pose information, thus improving the user experience.

[0086] In some embodiments, as shown in FIG3, generating each target video frame based on the shooting device pose information corresponding to each original video frame of the original video includes the following steps:

[0087] Step S210: Based on the pose information of the shooting device corresponding to each original video frame, determine the target direction information corresponding to each original video frame. The target direction information is used to characterize the direction from the shooting device to the target object in the world coordinate system when the original video frame is shot.

[0088] In step S210, the target direction information corresponding to each original video frame can be determined based on the pose information of the shooting device corresponding to each original video frame. This target direction is the direction the shooting device faces towards the target object in the world coordinate system when capturing each original video frame. This direction determines the framing angle of the shooting device around the target object and the position of the target object in the original video frame. For example, the direction of the shooting device towards the target object in the world coordinate system can be represented by a direction vector, and this direction vector can be used as the target direction information. Because the shooting device is in motion while the position of the target object is relatively fixed during the capture of the original video, the target direction information corresponding to each original video frame is different.

[0089] Step S220: Generate each target video frame based on each original video frame and its corresponding target direction information.

[0090] In step S220, since the target direction information corresponding to each original video frame determines the direction of the shooting device's framing angle around the target object and the location of the target object in the original video frame, the target video frame can be cropped according to the target direction information corresponding to the original video frame to generate a target video frame corresponding to the original video frame, so that the target video frame has a framing angle around the target object.

[0091] In this embodiment, the pose information of the shooting device corresponding to each original video frame is used to determine the target direction information corresponding to each original video frame. Based on each original video frame and its corresponding target direction information, target video frames are generated. This allows for the generation of target video frames based on the pose information of the shooting device, providing a basis for the generation of target videos. Using the direction from the shooting device to the target object in the world coordinate system during the capture of the original video frame as the basis for generating target video frames allows for the accurate and rapid determination of the shooting device's framing perspective around the target object and the location of the target object in the original video frame. This ensures the efficiency and accuracy of target video frame generation and improves the user experience.

[0092] In some embodiments, the target direction information corresponding to each original video frame is determined based on the shooting device pose information corresponding to each original video frame, including the following steps: Performing the following processing procedure as shown in Figure 4 on each original video frame:

[0093] Step S211: Based on the pose information of the shooting device corresponding to the original video frame, determine the mapping relationship between the world coordinate system and the preset coordinate system. The preset coordinate system is relatively fixed with respect to the shooting device.

[0094] In step S211, for each original video frame, the mapping relationship between the current world coordinate system and the preset coordinate system is determined based on the pose information of the shooting device corresponding to that original video frame. The world coordinate system is a global and fixed coordinate system used to describe the position and pose of all objects in the scene. In the world coordinate system, the positions of the shooting device, the target object, and all other objects are defined relative to a fixed reference point. The preset coordinate system is a local coordinate system fixed relative to the shooting device, used to describe the position and pose of objects in the scene relative to the shooting device. The preset coordinate system changes as the pose of the shooting device changes.

[0095] Since the pose information of the shooting device can represent the pose of the shooting device, the positional relationship between the origin of the world coordinate system and the preset coordinate system and the relative rotation relationship of the coordinate systems can be determined based on the pose information of the shooting device. The mapping relationship between the world coordinate system and the preset coordinate system at the current moment can be obtained, so that information representing direction, such as direction vectors, in the preset coordinate system can be converted to the world coordinate system.

[0096] Step S212: Based on the preset direction information and mapping relationship, determine the target direction information corresponding to the original video frame. The preset direction information is used to characterize the direction from the shooting device to the target object in the preset coordinate system.

[0097] In step S212, the preset direction information is used to characterize the direction from the shooting device to the target object in the preset coordinate system. Since the shooting device maintains a relatively fixed orientation towards the target object during the original video recording process (e.g., by rotating around it), and the preset coordinate system is relatively fixed with respect to the shooting device, the direction from the shooting device to the target object in the preset coordinate system is also relatively fixed. After determining the mapping relationship between the world coordinate system and the preset coordinate system, the direction from the shooting device to the target object, as represented by the preset direction information, can be transformed from the preset coordinate system to the world coordinate system based on this mapping relationship and the preset direction information. This determines the target direction information corresponding to each original video frame, providing a basis for the generation of the target video frame.

[0098] For example, when the shooting device rotates around the target object to shoot the original video, with the bottom of the shooting device facing the target object, the direction from the shooting device to the target object in the preset coordinate system represented by the preset direction information can always be represented by a direction vector of (0, 0, -1). According to the direction vector in the preset coordinate system and the mapping relationship between the world coordinate system and the preset coordinate system, the direction vector can be transformed from the preset coordinate system to the world coordinate system, thereby obtaining the direction vector representing the direction from the shooting device to the target object in the world coordinate system, so as to determine the target direction information.

[0099] In this embodiment, for each original video frame, the mapping relationship between the current world coordinate system and the preset coordinate system is determined based on the pose information of the shooting device corresponding to the original video frame. Then, based on the preset direction information and the mapping relationship, the target direction information corresponding to the original video frame is determined. This achieves the determination of the target direction information for each original video frame, providing a basis for the generation of the target video frame. Using the mapping relationship between the world coordinate system and the preset coordinate system, along with the preset direction information, as the basis for determining the target direction information leverages the relatively stable nature of the preset direction information during the original video shooting process. By transforming the coordinate system, the target direction information is determined quickly and accurately, ensuring the efficiency and accuracy of generating the target video frame and improving the user experience.

[0100] In other embodiments, based on the shooting device pose information corresponding to each original video frame, the target direction information corresponding to each original video frame is determined, including the following steps: Performing the following processing procedure on each original video frame:

[0101] Based on the pose information of the shooting device and the gravitational acceleration data corresponding to the original video frame, the centripetal force information is determined as the target direction information corresponding to the original video frame. The centripetal force information is used to characterize the acceleration direction of the shooting device after removing gravitational acceleration in the world coordinate system.

[0102] Each original video frame, during recording, contains not only the corresponding camera pose information but also corresponding gravitational acceleration data. This gravitational acceleration data can be acquired through an accelerometer integrated into the camera's inertial measurement unit (IMU) or a separate accelerometer, thus characterizing the camera's gravitational acceleration. For each original video frame, centripetal force information—the direction of acceleration of the camera in the world coordinate system after removing gravitational acceleration—can be determined based on the camera pose information and gravitational acceleration data. When the camera rotates around the target object, this acceleration direction represents the centripetal force direction of the camera relative to the target object, equivalent to the direction from the camera to the target object in the world coordinate system. Therefore, the centripetal force information can be used as target direction information, representing the direction from the camera to the target object.

[0103] In this embodiment, for each original video frame, centripetal force information is determined based on the pose information of the shooting device and the gravitational acceleration data corresponding to that original video frame. This centripetal force information is then used as the target direction information for that original video frame, thus determining the target direction information for each original video frame and providing a basis for generating the target video frame. Using centripetal force information as the target direction information leverages the characteristic that the centripetal force direction of the shooting device during the original video shooting process is consistent with the direction towards the target object. By determining the centripetal force direction, the target direction information is determined quickly and accurately, ensuring the efficiency and accuracy of generating the target video frame and improving the user experience.

[0104] In some embodiments, the preset coordinate system includes: a camera body coordinate system based on an inertial measurement unit, or a coordinate system having a fixed relative rotational extrinsic parameter to the camera body coordinate system based on an inertial measurement unit.

[0105] As mentioned earlier, the preset coordinate system is relatively fixed to the shooting device. The camera body coordinate system based on the inertial measurement unit can be used as the preset coordinate system to describe the position of the target object relative to the shooting device in the scene. Alternatively, any coordinate system with a fixed relative rotation extrinsic parameter to the camera body coordinate system can be used as the preset coordinate system. This transforms the fixed relationship between the camera body coordinate system and the shooting device into a fixed relationship between the camera body coordinate system and the camera body coordinate system with a fixed relative rotation extrinsic parameter, allowing the position of the target object relative to the shooting device to be described using this coordinate system.

[0106] In this embodiment, the camera body coordinate system based on the inertial measurement unit or a coordinate system with a fixed relative rotational extrinsic parameter is used as the preset coordinate system. This ensures a relatively fixed relationship between the preset coordinate system and the camera, making it easier to describe the position of the target object relative to the camera by using the camera body coordinate system or a coordinate system with a fixed relative rotational extrinsic parameter as the preset coordinate system. It also facilitates determining the mapping relationship between the world coordinate system and the preset coordinate system, providing a basis for determining the target direction information.

[0107] In some embodiments, the preset direction information includes: a preset direction vector pointing from the shooting device to the target object in a preset coordinate system, or a direction vector whose angle with the preset direction vector is less than a preset angle threshold, wherein the preset direction vector is a direction vector pointing from the shooting device to the target object in a preset coordinate system.

[0108] A preset direction vector pointing from the shooting device to the target object in a preset coordinate system can be used as preset direction information. The preset direction vector represents the direction from the shooting device to the target object in the preset coordinate system. Based on the mapping relationship between the world coordinate system and the preset coordinate system, the preset direction vector can be transformed from the preset coordinate system to the world coordinate system, thereby obtaining the target direction vector pointing from the shooting device to the target object in the world coordinate system. The target direction vector is then used as the target direction information to determine the target direction information.

[0109] Alternatively, a direction vector whose angle with the preset direction vector is less than a preset angle threshold (i.e., a direction vector close to the preset direction vector) can be used as preset direction information. This direction vector represents the approximate direction from the shooting device to the target object in the preset coordinate system. Based on the mapping relationship between the world coordinate system and the preset coordinate system, this direction vector can be transformed from the preset coordinate system to the world coordinate system, thus obtaining a direction vector in the world coordinate system whose angle with the target direction vector is less than the preset angle threshold. This direction vector is then used as the target direction information to determine the target direction information. The preset angle threshold can be set according to the accuracy requirements of the target direction information; for example, the preset angle threshold can be 10°.

[0110] In this embodiment, a preset direction vector pointing from the shooting device to the target object in a preset coordinate system, or a direction vector whose angle with the target object is less than a preset angle threshold, is used as preset direction information. This direction vector can be transformed to the world coordinate system based on the mapping relationship between the world coordinate system and the preset coordinate system, thereby determining the target direction information and providing a basis for determining the target video frame. Representing the preset direction information in the form of a direction vector makes the preset direction information intuitive, accurate, and easy to calculate coordinate system transformations, improving the efficiency and accuracy of determining the target direction information.

[0111] In some embodiments, as shown in FIG5, generating target video frames based on each original video frame and its corresponding target direction information includes the following steps:

[0112] S221: Determine the rotation axis information, which is used to characterize the direction of the rotation axis of the shooting device around the target object in the world coordinate system.

[0113] In step S221, when generating each target video frame based on the original video frame and the target direction information corresponding to the original video frame, it is necessary to first determine the rotation axis information. The rotation axis information can characterize the rotation axis direction of the shooting device around the target object in the world coordinate system. When the shooting device rotates around the target object, the rotation axis direction can be perpendicular to the direction from the shooting device to the target object. The rotation axis direction is relatively fixed in the world coordinate system, and its position can also reflect the position of the target object to a certain extent, so that the rotation axis information and the target direction information can jointly determine the position of the target object in the original video frame.

[0114] Step S222: Generate each target video frame based on each original video frame, the target direction information and rotation axis information corresponding to each original video frame.

[0115] In step S222, the rotation axis information and the target direction information can respectively represent the rotation axis direction of the shooting device around the target object in the world coordinate system and the direction of the shooting device to the target object. The two can clearly reflect the rotation state of the shooting device at the corresponding moment of the original video frame and jointly determine the position of the target object in the original video frame.

[0116] Therefore, for each original video frame, target video frames can be generated based on the original video frame, its corresponding target direction information, and rotation axis information. For example, the rendering trajectory of the target video frame relative to the original video frame, i.e., the displacement and rotation of the image frame, can be determined based on the target direction information and rotation axis information of the original video frame. Then, the image frame of the original video frame can be processed by displacement, rotation, and cropping based on this displacement and rotation, effectively changing the camera movement effect of the original video to obtain the corresponding target video frame.

[0117] In this embodiment, by determining the rotation axis information and generating target video frames based on each original video frame, the corresponding target direction information, and the rotation axis information, the generation of target video frames is achieved, providing a basis for the generation of target videos. Using the rotation axis information as the basis for generating target video frames allows the rotation state of the shooting device during the shooting of each original video frame to be accurately determined based on the rotation axis direction of the shooting device around the target object in the world coordinate system and the target direction information. This ensures that the framing angle of the generated target video frame can surround the target object, thereby generating a target video with a stable target locking effect.

[0118] In some embodiments, determining the rotation axis information includes the following steps: determining the rotation axis information based on the target direction information corresponding to each original video frame; or, determining the rotation axis information based on preset axis information; or, determining the rotation axis information based on the user's selection operation on the rotation axis of the shooting device.

[0119] When determining the rotation axis information, since the target direction information corresponding to each original video frame represents the direction from the shooting device to the target object in the world coordinate system, the target direction information corresponding to multiple original video frames can reflect the rotation process of the shooting device around the target object. Therefore, the rotation axis information can be determined based on the target direction information corresponding to each original video frame. For example, when the target direction information is a target direction vector, the rotation plane of the shooting device around the target object can be determined based on each target direction vector, and the direction perpendicular to the rotation plane can be taken as the rotation axis direction to achieve the determination of the rotation axis information.

[0120] Since the rotation of the shooting device around the target object is usually based on a direction perpendicular to the horizontal plane or other relatively fixed directions, the preset axis information can be used as the rotation axis information. That is, the rotation axis direction preset and defaulted by the shooting device can be used as the rotation axis direction required to generate the target video frame. This can reduce the amount of calculation of the rotation axis information in normal scenarios that match the preset axis information.

[0121] The system can also determine the rotation axis information based on the user's selection of the rotation axis of the shooting device, allowing the user to choose the rotation axis direction as needed. This selected rotation axis direction can then be used as the rotation axis direction required to generate the target video frame, reducing the computational load of the rotation axis information through user customization. For example, the user's selection of the rotation axis of the shooting device may include clicking or dragging the rotation axis selection control on the shooting device's user interface, or inputting the angle in the rotation axis angle input window on the shooting device's user interface.

[0122] In this embodiment, the rotation axis information is determined based on any one of the following: the target direction information corresponding to each original video frame, the preset axis information, and the user's selection of the rotation axis of the shooting device. This allows for the determination of the rotation axis information through calculations of the rotation process, default settings, and user-defined methods, providing a basis for the generation of the target video frame. Determining the rotation axis information based on the target direction information corresponding to each original video frame ensures its accuracy. Using the preset axis information or determining the rotation axis information based on the user's selection of the rotation axis of the shooting device enriches the methods for determining the rotation axis information and improves user operability, reduces the computational load, and enhances the user experience.

[0123] In some embodiments, determining rotation axis information based on the target direction information corresponding to each original video frame includes the following steps: determining a rotation axis information based on all target direction information.

[0124] When determining the rotation axis information based on the target direction information corresponding to each original video frame, a rotation axis information can be determined based on all the target direction information corresponding to all the original video frames, so as to characterize a rotation axis direction when the shooting device rotates around the target object in the world coordinate system.

[0125] Understandably, if the shooting device actually rotates around the target object around a fixed axis of rotation, the accuracy of the rotation axis information can be fully guaranteed. If the shooting device actually rotates around the target object around different axes of rotation successively due to the movement of the target object or the change of the rotation surface, multiple rotation cycles can be fitted to determine the rotation axis information that best matches the entire original video shooting process.

[0126] In this embodiment, a rotation axis is determined based on all target direction information, thus providing a basis for generating each target video frame. Limiting the number of rotation axis information to one ensures accuracy even when the shooting device rotates around the target object with a fixed axis. Furthermore, it allows for the determination of the rotation axis information that best matches the entire original video shooting process, even when the device rotates around the target object with different axes, thereby ensuring both efficiency and accuracy in generating target video frames.

[0127] In other embodiments, the rotation axis information is determined based on the target direction information corresponding to each original video frame, including the following steps: determining the rotation axis information based on multiple target direction information under different rotation periods, thereby obtaining multiple rotation axis information corresponding one-to-one with multiple rotation periods.

[0128] When determining the rotation axis information based on the target direction information corresponding to each original video frame, it is also possible to identify multiple rotation cycles of the shooting device around the target object during the original video shooting process, based on the target direction information corresponding to each original video frame. In different rotation cycles, the shooting device rotates around the target object with different rotation axes. For each rotation cycle, the rotation axis information corresponding to that rotation cycle can be determined based on the multiple target direction information, thus obtaining multiple rotation axis information corresponding one-to-one with multiple rotation cycles, representing the multiple rotation axis directions of the shooting device rotating around the target object in the world coordinate system.

[0129] It is understandable that by determining multiple rotation axis information corresponding to multiple rotation cycles one by one, the accuracy of each rotation axis information can be fully guaranteed. Furthermore, by using a series of rotation axis information and a series of target direction information corresponding to each rotation axis information, the framing perspective of the generated target video frame can be ensured to surround the target object.

[0130] In this embodiment, rotation axis information is determined based on multiple target direction information under different rotation cycles, resulting in multiple rotation axis information corresponding one-to-one with multiple rotation cycles. This determination of rotation axis information provides a basis for generating each target video frame. The multiple rotation axis information ensures the accuracy of each information, further guaranteeing that the framing perspective of the generated target video frame surrounds the target object, thus generating a target video with stable target locking effect.

[0131] In some embodiments, after determining the target direction information corresponding to each original video frame, the video processing method further includes the following step: smoothing the multiple target direction information.

[0132] After determining the target direction information corresponding to each original video frame, the determined target direction information can be smoothed to reduce data noise and fluctuations, facilitating the generation of target video frames based on the target direction information and further improving the camera movement effect of the target video. For example, the target direction vector representing the target direction information can be smoothed in vector form to achieve smoothing of the target direction information.

[0133] In this embodiment, after determining the target direction information corresponding to each original video frame, smoothing multiple target direction information can reduce data noise and fluctuations in the target direction information, improve the analyzability of the target direction information, reduce the computational load of generating target video frames, and enable the target video to have a smoother camera movement effect.

[0134] In some embodiments, after generating each target video frame based on each original video frame and its corresponding target direction information, the video processing method further includes the following step: smoothing the multiple target video frames.

[0135] After generating each target video frame, the generated target video frames can be smoothed to reduce image noise and fluctuations, facilitating the generation of the target video based on the target video frames and further improving the camera movement effect of the target video. For example, smoothing of multiple target video frames can be achieved through different methods such as pixel averaging, filtering, and Gaussian blurring.

[0136] In this embodiment, after generating each target video frame, smoothing the generated multiple target video frames can reduce image noise and playback, improve the composability of the target video frames, reduce the computational load of generating the target video, and enable the target video to have a smoother camera movement effect.

[0137] In some embodiments, the original video includes multiple original videos that correspond one-to-one with multiple lenses of the shooting device, and the video processing method further includes the following steps: performing a synthesis process on multiple target videos that correspond one-to-one with the multiple original videos to obtain a synthesized video with a second video playback speed.

[0138] When a shooting device is equipped with multiple lenses, each lens can capture a corresponding original video. Adjacent lenses have partially overlapping framing areas, resulting in original videos with some identical footage. After processing each original video using the aforementioned video processing method, a corresponding target video is obtained. Multiple target videos can be composited to combine their footage from the same moment into a single video with a larger framing, resulting in a composite video with a second playback speed. This composite video also exhibits a target object locking effect, enabling it to achieve both the stitching and fusion of multiple target videos and a bullet-time effect.

[0139] In this embodiment, by synthesizing multiple target videos that correspond one-to-one with multiple original videos, a synthesized video with a second video playback speed can be obtained. This allows the synthesized video to not only achieve the splicing and fusion of multiple target videos but also to have a bullet time effect, achieving seamless visual transitions and consistency between multiple shots, expanding the possibilities of shooting devices, and improving the user experience.

[0140] In one exemplary embodiment, a video processing method is provided, applied to a shooting device, as shown in FIG6. The video processing method includes the following steps:

[0141] Step S1: Obtain the pose information of the shooting device;

[0142] Step S2: In response to the shooting device's pose information meeting the preset shooting conditions, a video shooting command is generated;

[0143] Step S3: In response to the video recording command, acquire the original video, which has a first video playback speed;

[0144] Step S4: For each original video frame, based on the pose information of the shooting device corresponding to the original video frame, determine the mapping relationship between the world coordinate system and the preset coordinate system. The preset coordinate system is relatively fixed with respect to the shooting device.

[0145] Step S5: Based on the preset direction information and mapping relationship, determine the target direction information corresponding to the original video frame. The preset direction information is used to characterize the direction from the shooting device to the target object in the preset coordinate system.

[0146] Step S6: Smooth the direction information of multiple targets;

[0147] Step S7: Determine the rotation axis information. The rotation axis information is used to characterize the rotation axis direction of the imaging device around the target object in the world coordinate system.

[0148] Step S8: Based on each original video frame, the target direction information and rotation axis information corresponding to each original video frame, generate each target video frame. Multiple target video frames have different framing angles around the target object.

[0149] Step S9: Smooth multiple target video frames;

[0150] Step S10: Based on multiple target video frames, generate a target video with a second video playback speed, where the second video playback speed is less than the first video playback speed.

[0151] In this embodiment, by acquiring the original video, multiple target video frames can be generated from the original video, and a target video can be generated from these multiple target video frames, thus achieving automatic generation of the target video. The multiple target video frames have different framing perspectives around the target object, and the second playback speed of the target video is slower than the first playback speed of the original video, enabling the target video to provide a bullet-time effect with time slowdown and target locking. By generating target video frames with framing perspectives around the target object and slowing down video playback speed to generate a target video with bullet-time effects, the process of shooting and processing the original video is simplified, the generation cost of the target video is reduced, the final product quality of the target video is guaranteed, and the user experience is improved.

[0152] In one exemplary embodiment, a shooting device is provided. The shooting device may include, for example, an electronic device with shooting capabilities such as a mobile phone or a camera. Exemplarily, the shooting device can be a mobile phone or a camera with different fields of view, such as a traditional camera, a wide-angle camera, or a panoramic camera. The shooting device includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of any of the above-described video processing methods.

[0153] As shown in Figure 7, a structural block diagram of a computer device 100, which can serve as the imaging apparatus of this disclosure, will now be described. The computer device 100 includes a computing unit 101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 102 or a computer program loaded from a storage unit 108 into a random access memory (RAM) 103. The RAM 103 may also store various programs and data required for the operation of the computer device 100. The computing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0154] Multiple components in computer device 100 are connected to I / O interface 105, including: input unit 106, output unit 107, storage unit 108, and communication unit 109. Input unit 106 can be any type of device capable of inputting information into computer device 100. Input unit 106 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of computer device 100, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 107 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 108 may include, but is not limited to, a hard disk and an optical disk. Communication unit 109 allows computer device 100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0155] The computing unit 101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 101 performs the various methods and processes described above, such as video processing methods. For example, in some embodiments, the video processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 108. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer device 100 via ROM 102 and / or communication unit 109. When the computer program is loaded into RAM 103 and executed by the computing unit 101, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, the computing unit 101 may be configured to perform video processing methods by any other suitable means (e.g., by means of firmware).

[0156] Computer device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the video processing methods described above.

[0157] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0158] In the description of this specification, references to the terms "embodiment," "exemplary embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with implementation methods or examples that are included in at least one implementation method or example of this disclosure.

[0159] In this specification, the illustrative expressions of the terms used do not necessarily refer to the same implementation or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more implementations or examples.

[0160] It is understood that the terms "first," "second," etc., as used in this disclosure may be used to describe various structures, but these structures are not limited by these terms. These terms are only used to distinguish one structure from another.

[0161] In one or more accompanying drawings, the same elements are represented by similar reference numerals. For clarity, many parts in the drawings are not drawn to scale. Furthermore, certain well-known parts may not be shown. For simplicity, a structure obtained after several steps may be depicted in a single drawing. Many specific details of this disclosure, such as the structure, materials, dimensions, processing methods, and techniques of the devices, are described below to provide a clearer understanding of the disclosure. However, as those skilled in the art will understand, this disclosure may be implemented without adhering to these specific details.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure. Industrial applicability

[0163] The video processing method and shooting device provided in this disclosure, by acquiring the original video, can generate multiple target video frames from the original video, and generate a target video from the multiple target video frames, thus realizing the automatic generation of the target video. The multiple target video frames have different framing angles around the target object, and the second video playback speed of the target video is lower than the first video playback speed of the original video, enabling the target video to provide a bullet-time effect of time slowing down and target locking. By generating target video frames with framing angles around the target object and slowing down the video playback speed to generate a target video with bullet-time effects, the shooting and processing process of the original video is simplified, the generation cost of the target video is reduced, the final product quality of the target video is guaranteed, and the user experience is improved.

Claims

1. A video processing method, characterized in that, The video processing method includes: Acquire the original video, which has a first video playback speed; Based on the original video, multiple target video frames are generated, and the multiple target video frames have different framing angles around the target object; Based on the plurality of target video frames, a target video with a second video playback speed is generated, wherein the second video playback speed is less than the first video playback speed.

2. The video processing method according to claim 1, characterized in that, The field of view of the original video is less than 220°.

3. The video processing method according to claim 2, characterized in that, The field of view of the original video is greater than 70° and less than 120°.

4. The video processing method according to claim 1, characterized in that, The original video is a panoramic video with a spherical field of view, and the target video frame is a planar video frame determined based on the panoramic video frame.

5. The video processing method according to claim 1, characterized in that, During the recording of the original video, the shooting device rotates around the target object, and the framing angle of the shooting device is always oriented towards the side closest to the target object.

6. The video processing method according to claim 5, characterized in that, During the recording of the original video, the shooting device rotates following an auxiliary line or selfie stick connected to it to rotate around the target object.

7. The video processing method according to claim 1, characterized in that, Prior to acquiring the original video, the video processing method further includes: Obtain the pose information of the shooting device; In response to the shooting device's pose information meeting preset shooting conditions, a video shooting command is generated.

8. The video processing method according to claim 1, characterized in that, The field of view of the target video is smaller than that of the original video.

9. The video processing method according to claim 1, characterized in that, The framing range of the target video is smaller than the framing range of the original video.

10. The video processing method according to claim 1, characterized in that, The target object remains in the center area of ​​the video frame of the target video at all times.

11. The video processing method according to claim 1, characterized in that, The framing perspective of the multiple target video frames changes continuously, and the target video has a fixed scene effect; or... The original video is multiple, and the framing perspective of the multiple target video frames corresponding to each original video changes continuously. Moreover, the framing perspective of the multiple target video frames corresponding to different original videos revolves around the same target object, and the target video has a scene change effect.

12. The video processing method according to any one of claims 1 to 11, characterized in that, The process of generating multiple target video frames based on the original video includes: Target tracking is performed on the original video frames of the original video to obtain each target video frame, such that the framing perspective of each target video frame surrounds the target object.

13. The video processing method according to any one of claims 1 to 11, characterized in that, The process of generating multiple target video frames based on the original video includes: Based on the shooting device pose information corresponding to the original video frames of the original video, each of the target video frames is generated so that the framing angle of each of the target video frames surrounds the target object.

14. The video processing method according to claim 13, characterized in that, The pose information of the shooting device includes inertial measurement unit data acquired by the shooting device.

15. The video processing method according to claim 13, characterized in that, The generation of each target video frame based on the shooting device pose information corresponding to each original video frame of the original video includes: Based on the pose information of the shooting device corresponding to each original video frame, the target direction information corresponding to each original video frame is determined. The target direction information is used to characterize the direction from the shooting device to the target object in the world coordinate system when the original video frame is shot. Each target video frame is generated based on each original video frame and its corresponding target direction information.

16. The video processing method according to claim 15, characterized in that, The step of determining the target direction information corresponding to each of the original video frames based on the shooting device pose information corresponding to each of the original video frames includes: The following processing procedure is performed on each of the original video frames: Based on the pose information of the shooting device corresponding to the original video frame, the mapping relationship between the world coordinate system and the preset coordinate system is determined, and the preset coordinate system is relatively fixed relative to the shooting device; Based on preset direction information and the mapping relationship, the target direction information corresponding to the original video frame is determined. The preset direction information is used to characterize the direction from the shooting device to the target object in the preset coordinate system; or... Based on the pose information of the shooting device and the gravitational acceleration data corresponding to the original video frame, the centripetal force information is determined as the target direction information corresponding to the original video frame. The centripetal force information is used to characterize the acceleration direction of the shooting device after removing gravitational acceleration in the world coordinate system.

17. The video processing method according to claim 16, characterized in that, The preset coordinate system includes: The camera's coordinate system is based on an inertial measurement unit; or, A coordinate system with a fixed relative rotational external parameter to the camera coordinate system of the inertial measurement unit-based imaging device.

18. The video processing method according to claim 16, characterized in that, The preset direction information includes: The preset direction vector pointing from the shooting device to the target object in the preset coordinate system; or... A direction vector whose angle with a preset direction vector is less than a preset angle threshold, wherein the preset direction vector is the direction vector from the shooting device to the target object in the preset coordinate system.

19. The video processing method according to claim 15, characterized in that, The step of generating each target video frame based on each original video frame and its corresponding target direction information includes: Determine the rotation axis information, which is used to characterize the rotation axis direction of the imaging device around the target object in the world coordinate system; Each target video frame is generated based on each original video frame, the target direction information corresponding to each original video frame, and the rotation axis information.

20. The video processing method according to claim 19, characterized in that, The determination of the rotation axis information includes: The rotation axis information is determined based on the target direction information corresponding to each of the original video frames; or... The preset axial information is determined as the rotational axial information; or... The rotation axis information is determined based on the user's selection operation for the rotation axis of the shooting device.

21. The video processing method according to claim 20, characterized in that, Determining the rotation axis information based on the target direction information corresponding to each of the original video frames includes: Based on all the target direction information, determine one of the rotation axis information; or, Based on the target direction information under different rotation cycles, the rotation axis information is determined to obtain multiple rotation axis information corresponding one-to-one with the multiple rotation cycles.

22. The video processing method according to claim 15, characterized in that, After determining the target direction information corresponding to each of the original video frames, the video processing method further includes: Smoothing is performed on multiple target direction information; and / or, After generating each target video frame based on each original video frame and its corresponding target direction information, the video processing method further includes: The plurality of target video frames are smoothed.

23. The video processing method according to any one of claims 1 to 11, characterized in that, The original video includes multiple original videos that correspond one-to-one with multiple lenses of the shooting device, and the video processing method further includes: The target videos, each corresponding to one of the original videos, are synthesized to obtain a synthesized video with the playback speed of the second video.

24. A shooting device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, the processor is configured to acquire an original video, the original video having a first video playback speed; Based on the original video, multiple target video frames are generated, and the multiple target video frames have different framing angles around the target object; Based on the plurality of target video frames, a target video with a second video playback speed is generated, wherein the second video playback speed is less than the first video playback speed.

25. The shooting device according to claim 24, characterized in that, The field of view of the original video is less than 220°.

26. The shooting device according to claim 25, characterized in that, The field of view of the original video is greater than 70° and less than 120°.

27. The shooting device according to claim 24, characterized in that, The original video is a panoramic video with a spherical field of view, and the target video frame is a planar video frame determined based on the panoramic video frame.

28. The shooting device according to claim 24, characterized in that, During the recording of the original video, the shooting device rotates around the target object, and the framing angle of the shooting device is always oriented towards the side closest to the target object.

29. The shooting device according to claim 28, characterized in that, During the recording of the original video, the shooting device rotates following an auxiliary line or selfie stick connected to it to rotate around the target object.

30. The shooting device according to claim 24, characterized in that, The processor is configured to acquire the pose information of the shooting device; In response to the shooting device's pose information meeting preset shooting conditions, a video shooting command is generated.

31. The shooting device according to claim 24, characterized in that, The field of view of the target video is smaller than that of the original video.

32. The shooting device according to claim 24, characterized in that, The framing range of the target video is smaller than the framing range of the original video.

33. The shooting device according to claim 24, characterized in that, The target object remains in the center area of ​​the video frame of the target video at all times.

34. The shooting device according to claim 24, characterized in that, The framing perspective of the multiple target video frames changes continuously, and the target video has a fixed scene effect; or... The original video is multiple, and the framing perspective of the multiple target video frames corresponding to each original video changes continuously. Moreover, the framing perspective of the multiple target video frames corresponding to different original videos revolves around the same target object, and the target video has a scene change effect.

35. The shooting device according to any one of claims 24 to 34, characterized in that, The processor is configured to perform target tracking on the original video frames of the original video to obtain each target video frame, such that the framing perspective of each target video frame surrounds the target object.

36. The shooting device according to any one of claims 24 to 34, characterized in that, The processor is configured to generate each of the target video frames based on the shooting device pose information corresponding to the original video frames of the original video, such that the framing angle of each of the target video frames surrounds the target object.

37. The shooting device according to claim 36, characterized in that, The pose information of the shooting device includes inertial measurement unit data acquired by the shooting device.

38. The shooting device according to claim 36, characterized in that, The processor is configured to determine target direction information corresponding to each of the original video frames based on the pose information of the shooting device corresponding to each of the original video frames. The target direction information is used to characterize the direction from the shooting device to the target object in the world coordinate system when the original video frame is captured. Each target video frame is generated based on each original video frame and its corresponding target direction information.

39. The shooting device according to claim 38, characterized in that, The processor is configured to perform the following processing procedure on each of the original video frames: Based on the pose information of the shooting device corresponding to the original video frame, the mapping relationship between the world coordinate system and the preset coordinate system is determined, and the preset coordinate system is relatively fixed relative to the shooting device; Based on preset direction information and the mapping relationship, the target direction information corresponding to the original video frame is determined. The preset direction information is used to characterize the direction from the shooting device to the target object in the preset coordinate system; or... Based on the pose information of the shooting device and the gravitational acceleration data corresponding to the original video frame, the centripetal force information is determined as the target direction information corresponding to the original video frame. The centripetal force information is used to characterize the acceleration direction of the shooting device after removing gravitational acceleration in the world coordinate system.

40. The shooting device according to claim 39, characterized in that, The preset coordinate system includes: The camera's coordinate system is based on an inertial measurement unit; or, A coordinate system with a fixed relative rotational external parameter to the camera coordinate system of the inertial measurement unit-based imaging device.

41. The shooting device according to claim 39, characterized in that, The preset direction information includes: The preset direction vector pointing from the shooting device to the target object in the preset coordinate system; or... A direction vector whose angle with a preset direction vector is less than a preset angle threshold, wherein the preset direction vector is the direction vector from the shooting device to the target object in the preset coordinate system.

42. The shooting device according to claim 38, characterized in that, The processor is configured to determine rotation axis information, which is used to characterize the rotation axis direction of the imaging device around the target object in the world coordinate system; Each target video frame is generated based on each original video frame, the target direction information corresponding to each original video frame, and the rotation axis information.

43. The shooting device according to claim 42, characterized in that, The processor is configured to determine the rotation axis information based on the target direction information corresponding to each of the original video frames; or... The preset axial information is determined as the rotational axial information; or... The rotation axis information is determined based on the user's selection operation for the rotation axis of the shooting device.

44. The shooting device according to claim 43, characterized in that, The processor is configured to determine a rotation axis information based on all of the target direction information; or, Based on the target direction information under different rotation cycles, the rotation axis information is determined to obtain multiple rotation axis information corresponding one-to-one with the multiple rotation cycles.

45. The shooting device according to claim 38, characterized in that, The processor is configured to smooth the plurality of target orientation information; and / or to smooth the plurality of target video frames.

46. ​​The shooting device according to any one of claims 24 to 34, characterized in that, The original video includes multiple original videos that correspond one-to-one with multiple lenses of the shooting device. The processor is configured to perform synthesis processing on multiple target videos that correspond one-to-one with the multiple original videos to obtain a synthesized video with the second video playback speed.

Images