Video processing method, device, computer equipment and storage medium

Abstract

The present application relates to a video processing method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining the original video and camera motion data collected by the camera during the movement; determining the target time when the camera reaches the target point during the movement according to the camera motion data; extracting the target video frame corresponding to the target time from the original video; rotating the target video frame according to the target time and the preset duration to obtain a rotated video. The use of this method can improve the visual effect of the picture.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the development of computer technology, panoramic cameras are being used more and more widely. Panoramic cameras can shoot panoramic videos. The advantage of panoramic videos is that they have a very wide viewing angle. For example, panoramic videos can cover a 360-degree viewing angle.

[0003] However, the display shows a flat image, so more and more people are studying how to fully and effectively display 360-degree panoramic videos to effectively improve the user experience. Summary of the invention

[0004] Based on this, it is necessary to provide a video processing method, device, computer equipment, and computer-readable storage medium to address the technical problem of how to fully and effectively display panoramic videos.

[0005] In a first aspect, the present application provides a video processing method. The method comprises:

[0006] Obtain the original video and camera motion data collected by the camera during movement;

[0007] Determining, based on the camera motion data, a target time at which the camera reaches a target point during the motion process;

[0008] Extracting a target video frame corresponding to the target moment from the original video;

[0009] According to the target time and the preset duration, the target video frame is rotated to obtain a rotated video.

[0010] In a second aspect, the present application further provides a video processing device. The device comprises:

[0011] An acquisition module is used to acquire the original video and camera motion data collected by the camera during the movement;

[0012] A determination module, for the camera motion data, to determine a target time when the camera reaches a target point during the motion;

[0013] An extraction module, used for extracting a target video frame corresponding to the target moment from the original video;

[0014] The rotation processing module is used to perform rotation processing on the target video frame according to the target time and the preset duration to obtain a rotated video.

[0015] In one embodiment, the apparatus further comprises:

[0016] A calculation module, used for calculating the yaw angle and the pitch angle corresponding to each time point of the target video frame within the preset time length according to the target time and the preset time length;

[0017] The rotation processing module is further used to perform rotation processing on the target video frame based on the yaw angle and the pitch angle to obtain a rotated video.

[0018] In one embodiment, the determining module is further used to:

[0019] Determine, according to the camera motion data, a first acceleration and a second acceleration of the camera during the motion process;

[0020] A target time when the camera reaches a target point during the movement is determined according to a first time corresponding to the first acceleration and a second time corresponding to the second acceleration.

[0021] In one embodiment, the rotation processing module is further used to:

[0022] Taking the target time as the rotation start time, and determining the rotation end time according to the rotation start time and the preset duration;

[0023] Determine the rotation period according to the rotation start time and the rotation end time;

[0024] Calculating the ratio of each time point within the preset time length to the rotation period;

[0025] Based on the ratio and the initial yaw angle, calculating the yaw angle of the target video frame corresponding to each time point within the preset duration;

[0026] Based on the ratio and the initial pitch angle, the pitch angle of the target video frame corresponding to each time point within the preset duration is calculated.

[0027] In one embodiment, the time points within the preset duration constitute a time point set, and the time point set includes a first time point set and a second time point set; the first time point set consists of first time points before the target video frame is rotated to a first target position; the second time point set consists of second time points between the first target position and the second target position when the target video frame is rotated; the rotation processing module is further used to:

[0028] Based on the yaw angle and the pitch angle corresponding to the first time point, rotating the target video frame toward the upper left direction or the upper right direction;

[0029] Copying the target video frames rotated at each of the first time points to obtain a first video frame sequence;

[0030] When the target video frame rotates to the first target position, based on the yaw angle and the pitch angle corresponding to the second time point, the target video frame is rotated toward the lower right direction or the lower left direction until it rotates to the second target position;

[0031] Copying the target video frame rotated at each of the second time points to obtain a second video frame sequence;

[0032] A rotated video is generated based on the first video frame sequence and the second video frame sequence.

[0033] In one embodiment, the time points within the preset duration constitute a time point set; the time point set includes a third time point set; the third time point set is composed of each third time point in the process of causing the target video frame to rotate horizontally at the highest point of the motion process; the rotation processing module is further used to:

[0034] When the target video frame rotates to the highest point, the target video frame is rotated in the horizontal direction based on the yaw angle and the pitch angle corresponding to the third time point; when the target video frame rotates in the horizontal direction, the rotation speed component in the vertical direction is zero.

[0035] In one embodiment, the apparatus further comprises:

[0036] A recognition module, used to identify the tracking target in the pre-collected video;

[0037] The acquisition module is further used to acquire the tracking data corresponding to the tracking target;

[0038] An adjustment module, configured to adjust the viewing angle of the original video according to the tracking data to obtain the adjusted original video;

[0039] The extraction module is further used to extract the target video frame corresponding to the target moment from the adjusted original video.

[0040] In one embodiment, the apparatus further comprises:

[0041] An inserting module, used for inserting the rotated video into the original video according to the target moment to obtain a special effect video;

[0042] The playing module is used to play the special effect video in response to a playing instruction for the special effect video.

[0043] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0044] Obtain the original video and camera motion data collected by the camera during movement;

[0045] Determining, based on the camera motion data, a target time at which the camera reaches a target point during the motion process;

[0046] Extracting a target video frame corresponding to the target moment from the original video;

[0047] According to the target time and the preset duration, the target video frame is rotated to obtain a rotated video.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Obtain the original video and camera motion data collected by the camera during movement;

[0050] Determining, based on the camera motion data, a target time at which the camera reaches a target point during the motion process;

[0051] Extracting a target video frame corresponding to the target moment from the original video;

[0052] According to the target time and the preset duration, the target video frame is rotated to obtain a rotated video.

[0053] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0054] Obtain the original video and camera motion data collected by the camera during movement;

[0055] Determining, based on the camera motion data, a target time at which the camera reaches a target point during the motion process;

[0056] Extracting a target video frame corresponding to the target moment from the original video;

[0057] According to the target time and the preset duration, the target video frame is rotated to obtain a rotated video.

[0058] The above-mentioned video processing method, device, computer equipment, storage medium and computer program product obtain the original video and camera motion data collected by the camera during the movement process, and determine the target time when the camera reaches the target point during the movement process according to the camera motion data. The target video frame corresponding to the target time is extracted from the original video. Therefore, the target video frame can be accurately determined according to the camera motion data, and the accuracy of the determined target video frame is guaranteed. Then, according to the target time and the preset duration, the target video frame is rotated to obtain a rotated video. Since the computer equipment can accurately determine the target point reached by the camera during the movement according to the camera motion data, the efficiency of determining the target point is improved, and the visual effect of the rotated video is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A diagram showing an application environment of a video processing method in an embodiment;

[0060] Figure 2 is a schematic flow chart of a video processing method in one embodiment;

[0061] Figure 3 A schematic diagram of rotating a target video frame rendered in a spherical model in one embodiment;

[0062] Figure 4 A schematic diagram of a flow chart of a method for determining a target time in one embodiment;

[0063] Figure 5 A schematic diagram of a flow chart of a method for calculating a yaw angle and a pitch angle in one embodiment;

[0064] Figure 6 A schematic diagram of a flow chart of a method for adjusting the viewing angle of an original video in one embodiment;

[0065] Figure 7 is a flow chart of a video processing method in another embodiment;

[0066] Figure 8 is a structural block diagram of a video processing device in one embodiment;

[0067] Fig. 9 is a structural block diagram of a video processing device in one embodiment;

[0068] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] The video processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 obtains the original video and camera motion data collected by the camera during the movement; according to the camera motion data, the target time when the camera reaches the target point during the movement is determined; the target video frame corresponding to the target time is extracted from the original video; according to the target time and the preset duration, the yaw angle and pitch angle corresponding to each time point of the target video frame within the preset duration are calculated; based on the yaw angle and the pitch angle, the target video frame is rotated to obtain a rotated video. Among them, the terminal 102 can be but is not limited to a camera, a camera, a smart phone or a portable wearable device. The camera can be an ordinary camera, a panoramic camera or a sports camera, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0071] It should be noted that how to fully and effectively display the 360-degree panoramic video.

[0072] The method used may be: according to a frame of the video shot by the panoramic camera, a rotated video is generated. For example, by manually observing the video shot by the panoramic camera, a video frame shot by the panoramic camera at the target point is found, and then the video frame determined by human eye observation is rotated to fully display the picture information of the target point (such as the highest point), and generate a picture with a frozen visual effect.

[0073] In the above operation process, users not only need to learn a lot of panoramic video editing knowledge and operations, such as key frames, rotation or dot marking, which brings a lot of learning and labor costs; but also the confirmed target point is inaccurate, resulting in poor visual effects of the resulting rotated video (or the original video inserted into the rotated video). For example, when a user performs a video editing of a throw shot (throwing the camera up and shooting a video), the video frame of the highest point needs to be determined.

[0074] If the time point of the determined video frame is not the highest point, it means that the highest and widest field of view brought by the highest point cannot be obtained, and it also means that the camera will continue to rise in the next frame of the original video, while the end of the rotated video frame is falling. When the rotated video is inserted into the original video, it will produce a very large sense of frustration, resulting in poor effect of the original video after the rotated video is inserted.

[0075] Regarding the technical problem of how to obtain a better rotated video (or insert the original video into the rotated video), in one embodiment, as Figure 2 As shown, a video processing method is provided, which is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0076] S202, obtaining the original video and camera motion data collected by the camera during the motion process.

[0077] Among them, the movement is the movement of the camera leaving the initial position at a certain initial velocity under the action of gravity or other forces, including upward throwing motion or horizontal motion, etc. The upward throwing motion may be, for example, vertical upward throwing motion, vertical downward throwing motion, horizontal throwing motion or oblique throwing motion, etc. For example, when the camera is thrown vertically by the user, the movement process of the camera includes: first moving diagonally upward until the vertical speed is zero, and then moving downward until it falls at the target position. For another example, when the camera is ejected horizontally by an elastic component (e.g., a spring), the movement process of the camera includes: first moving away from the starting point to the farthest end, and then returning to the starting point under the action of the elastic force. The camera collects raw data during the movement, and the collected raw data includes the original video and the camera motion data. The original video is the video collected by the camera during the movement, which may be a panoramic video. A panoramic video is a video with a viewing angle greater than the normal effective viewing angle of the human eye, including a black and white video or a color video, etc. For example, a panoramic video is a video with a viewing angle greater than 90 degrees in the horizontal direction and a viewing angle greater than 70 degrees in the vertical direction.

[0078] S204: Determine, according to the camera motion data, a target time at which the camera reaches the target point during the motion process.

[0079] Among them, the camera motion data is data recording the camera motion, which can be data collected by the gyroscope in the camera, including the acceleration of the camera in the three directions of X-axis, Y-axis and Z-axis, or can also be the acceleration obtained by synthesizing the acceleration in the three directions of X-axis, Y-axis and Z-axis. The target point can be any point reached by the camera during the motion process, for example, the target point can be the highest point reached by the camera based on the upward motion. For another example, the target point can be a point where the vertical speed of the camera is zero. For another example, the target point can be a point where the horizontal speed of the camera is zero. The target moment is the moment when the camera reaches the target point. For example, the target moment is the 3rd minute and 4th second of the original video. For another example, the target moment is 11 hours, 3 minutes and 15 seconds. S206, extracting the target video frame corresponding to the target moment from the original video.

[0080] The target video frame is a video frame captured when the camera moves to the target point. For example, the target video frame is a video frame captured when the camera moves to the highest point. For another example, the target video frame is a video frame captured when the camera moves to a vertical speed of zero. For another example, the target video frame is a video frame captured when the camera moves to a horizontal speed of zero.

[0081] In one embodiment, S206 specifically includes: rendering the original video according to the three-dimensional spherical model, and extracting the target video frame corresponding to the target moment from the spherical panoramic video obtained after rendering.

[0082] S208, rotating the target video frame according to the target time and the preset duration to obtain a rotated video.

[0083] The preset duration is the duration of the generated video effect, which can be any value. For example, the preset duration is 8 seconds, 6 seconds, or 0.1 minutes. The rotation processing is a processing method that rotates the viewing angle corresponding to the target video frame. For example, Figure 3 As shown, the rotation processing can be a processing method for rotating the angle of the virtual camera at the center of the spherical model to look at the target video frame. When the terminal rotates the target video frame, the target video frame can have a rotation speed component in the horizontal direction or the vertical direction, or the target video frame can have a rotation speed component in both the horizontal direction and the vertical direction.

[0084] In the above embodiment, the original video captured by the camera during the movement is obtained, and the target time when the camera reaches the target point during the movement is determined according to the camera motion data extracted from the original video, so that the target point can be accurately determined according to the camera motion data, thereby improving the accuracy of determining the target point. The target video frame corresponding to the target time is extracted from the original video. Then, according to the target time and the preset duration, the target video frame is rotated to obtain a rotated video. Since the computer device can accurately determine the target point reached by the camera during the movement according to the camera motion data, the efficiency and accuracy of determining the target point are improved, thereby accurately obtaining the target video frame and improving the visual effect of the rotated video.

[0085] In one embodiment, S208 specifically includes: calculating the yaw angle and pitch angle corresponding to each time point of the target video frame within the preset time length according to the target time and the preset time length; rotating the target video frame based on the yaw angle and the pitch angle to obtain a rotated video.

[0086] The yaw angle is the angle of rotation of the viewing angle corresponding to the target video frame around the Z axis. For example, the yaw angle is 30 degrees, 45 degrees, or 60 degrees. The pitch angle is the angle of rotation of the viewing angle corresponding to the target video frame around the X axis. For example, the pitch angle is 15 degrees, 20 degrees, or 25 degrees. Figure 3 As shown, the target video frame is a 3D panoramic video frame rendered in a spherical model, and the viewing angle corresponding to the target video frame is the viewing angle of the target video frame observed by a virtual camera in the spherical model. When the viewing angle corresponding to the target video frame moves from point A to point B, the yaw angle and the pitch angle change accordingly.

[0087] The terminal calculates the yaw angle and pitch angle corresponding to each time point of the target video frame within the preset time length according to the target time and the preset time length. For example, at the target time (t0), the viewing angle corresponding to the target video frame looks at point A, and then the terminal rotates the viewing angle corresponding to the target video frame within the preset time length (assuming it is L), and the rotation end time point is calculated according to the target time and the preset time length as t3=t0+L. The terminal makes the viewing angle corresponding to the target video frame look at point B at time point t1, at point C at time point t2, and at point D at time point t3. The terminal can calculate the yaw angle and pitch angle corresponding to the target video frame at time points t1, t2, and t3 respectively according to t0 and t3.

[0088] In the above embodiment, the terminal calculates the yaw angle and pitch angle corresponding to each time point of the target video frame within the preset time length according to the target moment and the preset time length; based on the yaw angle and the pitch angle, the target video frame is rotated to obtain a rotated video. Therefore, the yaw angle and the pitch angle can be accurately determined according to the target moment and the preset time length, the accuracy of the yaw angle and the pitch angle is guaranteed, and the stability of the rotation effect is improved; and the target point is determined through motion data, so as to combine with the subsequent video frame rotation operation to realize automatic video editing, without the user having to learn too much knowledge of panoramic video editing, which effectively reduces the user's learning cost for panoramic video editing.

[0089] In one embodiment, Figure 4 As shown, S204 specifically includes the following steps:

[0090] S402: Determine a first acceleration and a second acceleration of the camera during movement according to the camera movement data.

[0091] Among them, the first acceleration is the maximum acceleration of the camera during the movement. For example, when the camera moves based on the user's throwing action, the first acceleration is the acceleration of the camera at the moment of being thrown up, and the direction of the first acceleration is the same as the direction of movement of the camera, that is, the first acceleration is a positive value. For another example, when the camera is horizontally ejected by the connected elastic component (such as a spring), the first acceleration is the acceleration of the camera at the moment of being ejected. The second acceleration is the minimum acceleration of the camera during the movement, and the direction of the second acceleration is opposite to the direction of movement of the camera, that is, the second acceleration is a negative value. For example, when the camera moves based on the user's throwing action, the second acceleration is the acceleration at the end of the camera's movement process. For another example, when the camera is horizontally ejected by the connected elastic component (such as a spring), the second acceleration is the acceleration of the camera when it returns to the starting point based on the pulling force of the elastic component.

[0092] In one embodiment, the first acceleration and the second acceleration are accelerations synthesized from accelerations in three directions: X-axis, Y-axis, and Z-axis.

[0093] S404: Determine a target time when the camera reaches the target point during the movement according to a first time corresponding to the first acceleration and a second time corresponding to the second acceleration.

[0094] The first moment is the moment when the camera has a first acceleration. For example, when the camera moves based on the user's throwing action, the first moment is the moment when the camera is thrown up. For another example, when the camera is horizontally popped up by an elastic component (such as a spring), the first moment is the moment when the camera is popped up. The second moment is the moment when the camera has a second acceleration. For example, when the camera moves based on the user's throwing action, the second moment is the moment when the camera movement process ends. For another example, when the camera is horizontally popped up by the connected elastic component (such as a spring), the second moment is the moment when the camera returns to the starting point based on the pulling force of the elastic component.

[0095] In one embodiment, the target point is the highest point of the camera based on the throwing action, or the target point can also be the farthest point of the camera when it is horizontally ejected by the elastic component. The target time is the middle time between the first time and the second time, and the terminal calculates the target time according to formula (1). Among them, T0 is the target time, T1 is the first time, and T2 is the second time.

[0096] T0=(T2+T1) / 2 (1)

[0097] In the above embodiment, the first acceleration and the second acceleration of the camera during the movement are determined according to the camera motion data. The target moment when the camera reaches the target point during the movement is determined according to the first moment corresponding to the first acceleration and the second moment corresponding to the second acceleration. The target moment when the camera reaches the target point is accurately determined according to the first acceleration and the second acceleration, so that the target video frame can be accurately extracted from the original video according to the target moment, thereby ensuring the visual effect of the obtained special effect picture.

[0098] In one embodiment, the target point is the highest point reached by the camera during the upward motion. Since the camera will not be blocked when reaching the highest point, it can be shot with the best visual field, and the target video frame corresponding to the highest point has the best visual effect of the picture, so the visual effect of the rotated video obtained by rotating the target video frame is good, which improves the visual effect of the rotated video. In addition, when the highest point is determined manually, the determined highest point may have an error and is not the actual highest point of the camera movement. If the target video frame corresponding to the manually determined highest point is rotated, the target video frame is first rotated upward and then rotated downward, and the rotated video obtained by the rotation process is inserted into the original video to obtain a special effect video. In the special effect video, when the rotating video ends rotating downward and returns to the original video, the original video will continue to rotate upward, that is, the rotating video cannot be smoothly combined with the original video, resulting in poor visual effects. In the present application, the highest point of the camera movement process can be accurately determined by camera motion data, so that the rotating video can be smoothly combined with the original video, which improves the visual effect of the video special effect.

[0099] In one embodiment, Figure 5 As shown, S208 specifically includes the following steps:

[0100] S502: Taking the target time as the rotation start time, and determining the rotation end time according to the rotation start time and a preset duration.

[0101] The rotation start time is the time when the target video frame starts to be rotated. For example, the terminal can reset the time of the rotation start time to zero, so that the rotation start time is zero. For another example, the terminal can determine the rotation start time according to the time when the target video frame appears in the original video. Specifically, if the target video frame appears at the 6.05th second in the original video, the rotation start time is determined to be the 6.05th second.

[0102] In one embodiment, S502 specifically includes: calculating the sum of the rotation start time and the preset duration, and using the obtained sum as the rotation end time.

[0103] S504, determining a rotation period according to the rotation start time and the rotation end time.

[0104] The rotation start time and the rotation end time are defined as a total cycle, and a total cycle can include multiple rotation cycles, and the durations of the multiple rotation cycles can be the same or different, which are specifically set according to the actual rotation effect requirements. For example, when the rotation is divided into two sections, a rotation cycle can be equal to half of the total cycle, and when the rotation is divided into three sections, the rotation cycle can be one-third of the total cycle. It can be understood that the durations of the rotation cycles may also be unequal.

[0105] In one embodiment, S504 specifically includes: the terminal calculates the sum of the rotation start time and the rotation end time, and then calculates the ratio of the sum to 2 to obtain the rotation period. Specifically, the terminal calculates the rotation period according to formula (2), where halftime is the rotation period, endtime is the rotation end time, and startime is the rotation start time.

[0106] halftime=(endtime-startime) / 2 (2)

[0107] S506, calculating the ratio of each time point within a preset time length to the rotation period.

[0108] The terminal calculates the ratio of the rotation period to each time point within the preset duration, and the calculated ratio is t / halftime, where t can be each time point within the preset duration.

[0109] S508: Calculate the yaw angle of the target video frame corresponding to each time point within a preset time length based on the ratio and the initial yaw angle.

[0110] The initial yaw angle is a preset yaw angle, for example, the initial yaw angle may be the yaw angle corresponding to the viewing angle of the target video frame at the start of the rotation. In one embodiment, S508 specifically includes: the terminal calculates the product of the ratio and 2π, and then adds the obtained product to the initial yaw angle to obtain the yaw angle corresponding to each time point of the target video frame within the preset duration. Specifically, the terminal calculates the yaw angle according to formula (3), wherein Yaw0 is the initial yaw angle, t may be each time point within the preset duration, and halftime is the rotation period.

[0111] Yaw=Yaw0+(t / halftime)×2π (3)

[0112] S510, based on the ratio and the initial pitch angle, calculating the pitch angle corresponding to each time point of the target video frame within a preset time length.

[0113] The initial pitch angle is a preset pitch angle, for example, the initial pitch angle is the pitch angle corresponding to the viewing angle of the target video frame at the start of the rotation. In one embodiment, S510 specifically includes: the terminal calculates the ratio and The sine of the product is then multiplied by a preset parameter (the preset parameter may be, for example, 0.058). Finally, the value obtained by multiplying the preset parameter is added to the initial pitch angle to obtain the pitch angle corresponding to each time point in the preset duration of the target video frame. Specifically, the terminal calculates the pitch angle according to formula (4), where pitch0 is the initial pitch angle, t may be each time point in the preset duration, and halftime is the rotation period.

[0114]

[0115] In the above embodiment, the target moment is taken as the rotation start moment, and the rotation end moment is determined according to the rotation start moment and the preset duration. The rotation period is determined according to the rotation start moment and the rotation end moment, and the ratio of each time point within the preset duration to the rotation period is calculated. Then, based on the ratio and the initial yaw angle, the yaw angle corresponding to each time point of the target video frame within the preset duration is calculated; based on the ratio and the initial pitch angle, the pitch angle corresponding to each time point of the target video frame within the preset duration is calculated. Thus, the target video frame can be rotated at a uniform speed according to the accurately determined yaw angle and pitch angle, thereby ensuring the stability of the rotation effect and improving the visual effect of the rotating video. In addition, when the target video frame is rotated according to the pitch angle and the yaw angle, the target video frame can be rotated to the upper left or upper right direction at a uniform speed through the synergistic effect of the pitch angle and the yaw angle, and then rotated to the lower left or lower right when rotating to the target position, thereby ensuring that the rotating video has the best visual effect.

[0116] In one embodiment, each time point within a preset duration constitutes a time point set, and the time point set includes a first time point set and a second time point set; the first time point set consists of each first time point before the target video frame is rotated to the first target position; the second time point set consists of the second time point between the first target position and the second target position; S210 specifically includes: based on the yaw angle and pitch angle corresponding to the first time point, rotating the target video frame to the upper left or upper right direction; copying the target video frame rotated at each first time point to obtain a first video frame sequence; when the target video frame is rotated to the first target position, based on the yaw angle and pitch angle corresponding to the second time point, rotating the target video frame to the lower right or lower left direction until it is rotated to the second target position; copying the target video frame rotated at each second time point to obtain a second video frame sequence; generating a rotated video based on the first video frame sequence and the second video frame sequence.

[0117] The first target position is a preset rotation position, which can be a plane where the pitch angle of the target video frame corresponding to the viewing angle is offset upward by 0.058 radians relative to the initial pitch angle. The second target position is the position where the viewing angle of the target video frame points to the tracking target.

[0118] The terminal first rotates the target video frame to the upper left or upper right direction based on the yaw angle and pitch angle corresponding to each first time point in the first time point set, that is, the view angle corresponding to the target video frame is rotated to the upper left or upper right direction, until the view angle of the target video frame is rotated to the plane of the first target position. At each first time point of each rotation, the terminal copies the target video frame rotated at that time point. The video frames copied at each first time point are combined to obtain a first video frame sequence. Then, based on the yaw angle and pitch angle corresponding to each second time point in the second time point set, the terminal rotates the target video frame to the lower right or lower left direction until it is rotated to the second target position. At each second time point of each rotation, the terminal copies the target video frame rotated at that time point. The video frames copied at each second time point are combined to obtain a second video frame sequence. Finally, the terminal generates a rotated video based on the first video frame sequence and the first video frame sequence.

[0119] In the above embodiment, the terminal first rotates the target video frame to the upper left or upper right direction according to the yaw angle and the pitch angle, and then rotates the target video frame to the lower right or lower left direction, thereby ensuring the stability of the rotation effect and improving the visual effect of the rotating video. In addition, the terminal rotates the target video frame to the upper left or upper right first according to the yaw angle and the pitch angle, and then rotates it to the lower right or lower left, that is, the target video frame is first rotated obliquely upward and then rotated obliquely downward, so that the effect of looking around can be produced through oblique rotation. Compared with another solution for realizing looking around: vertically rotating upward, then rotating horizontally, and finally rotating vertically downward (there is a setback in the connection between the three stages), the visual effect of the rotation in this embodiment is smoother, which improves the visual effect of the rotation.

[0120] In one embodiment, the time point set composed of time points within a preset duration also includes a third time point set; the third time point set is composed of time points that cause the target video frame to rotate horizontally at the highest point of the movement process; when the target video frame rotates to the highest point, based on the yaw angle corresponding to the third time point, the target video frame is rotated in the horizontal direction; when the target video frame rotates in the horizontal direction, the rotation speed component in the vertical direction is zero.

[0121] The terminal can rotate the target video frame to a preset target position at the end of the rotation phase corresponding to the third time point. For example, the preset target position can be a position where the view angle corresponding to the target video frame is rotated from the first target position by a preset angle. For example, the target video frame rotates in the horizontal direction from the first target position, and reaches the preset target position when the view angle rotates by a preset angle such as 180 degrees, 360 degrees, or 90 degrees.

[0122] In one embodiment, the terminal can rotate the target video frame vertically upward or diagonally upward (upper left or upper right) to the highest point, then keep the rotation speed component in the vertical direction to zero and rotate horizontally, and at the end of the rotation phase corresponding to the third time point, rotate the target video frame vertically downward or diagonally downward (lower left or lower right). The terminal rotates the target video frame horizontally at the highest point, which can produce a visual effect of looking around.

[0123] When the terminal rotates the target video frame to the highest point of the motion process, the rotation speed component of the target video frame in the vertical direction is made zero, and the target video frame is rotated in the horizontal direction. For example, the target video frame is rotated in the horizontal direction for one circle. Then, the target video frame is rotated to the lower right or lower left direction.

[0124] In the above embodiment, when the target video frame is rotated to the first target position, the target video frame is rotated in the horizontal direction based on the yaw angle and pitch angle corresponding to the third time point, and when the target video frame is rotated to the third target position, the target video frame is rotated to the lower right or lower left direction based on the yaw angle and pitch angle corresponding to the second time point until it is rotated to the second target position. In this way, the rotation effect of the target video frame can be enriched and the visual effect of the rotated video can be improved.

[0125] In one embodiment, Figure 6 As shown, before S206, S602-S606 are also included, and S206 specifically includes S608.

[0126] S602: Identify a tracking target in the captured video.

[0127] The tracking target is the target pointed to by the perspective corresponding to the video, which may be a person or an object, for example, the tracking target may be a user who throws the camera upward. In one embodiment, S602 specifically includes: the terminal starts to collect video in response to the video collection instruction, and identifies the tracking target in the collected video through image recognition technology.

[0128] S604: Acquire tracking data corresponding to the tracking target.

[0129] The tracking data is data used to represent the position of the tracking target in the video screen. For example, the tracking data is the position coordinates of the tracking target in the spherical model. For example, the tracking data is (x, y, z), where x, y, and z are the position coordinates of the tracking target on the x-axis, y-axis, and z-axis, respectively.

[0130] S606, adjusting the viewing angle of the original video according to the tracking data to obtain an adjusted original video.

[0131] The terminal adjusts the viewing angle of the original video according to the tracking data so that the viewing angle of each video frame in the original video points to the tracking target. Specifically, the terminal adjusts the viewing angle of the original video so that the viewing angle of each video frame in the original video points to the position indicated by the tracking data.

[0132] S608, extracting a target video frame corresponding to the target moment from the adjusted original video.

[0133] The terminal adjusts the viewing angle of the original video according to the tracking data, and extracts the target video frame corresponding to the target moment from the adjusted original video.

[0134] In the above embodiment, the tracking target is identified in the captured video, the tracking data corresponding to the tracking target is obtained, the viewing angle of the original video is adjusted according to the tracking data, and the target video frame corresponding to the target moment is extracted from the adjusted original video. Thus, the viewing angle in the adjusted original video can always look at the tracking target, which has a better visual effect.

[0135] In one embodiment, after S210, the method further includes: inserting the rotated video into the original video according to the target moment to obtain a special effect video; and playing the special effect video in response to a play instruction for the special effect video.

[0136] Among them, the special effects video is a video with special visual effects. For example, the special effects video is a video with a solidification visual effect. The solidification visual effect is a visual effect in which the picture taken when the camera moves to the target point is repeatedly played for a period of time, and the perspective of the picture is rotated during the repeated playback. For example, the solidification visual effect can be a high-altitude solidification visual effect. The high-altitude solidification visual effect is a visual effect in which the picture taken when the camera moves to the highest point based on the throwing action is repeatedly played for a period of time, and the perspective of the picture is rotated during the repeated playback. The target moment can be the moment when the camera reaches the target point. The terminal inserts the rotating video into the target video frame corresponding to the target moment in the original video, generating a visual effect that the picture is solidified starting from the target moment.

[0137] In the above embodiment, the rotated video is inserted into the original video according to the target time to obtain the special effect video. In response to the play instruction for the special effect video, the special effect video is played, which improves the rotation stability of the special effect video and ensures the visual effect.

[0138] In one embodiment, the terminal is a panoramic camera with a wide-angle lens. When the panoramic camera receives a video recording instruction triggered by the user, it automatically selects a tracking target and obtains tracking data corresponding to the tracking target. When the panoramic camera is thrown into the air by the user, the panoramic camera shoots the original video based on the process of the throwing action. Then the viewing angle of the original video is adjusted according to the tracking data so that the viewing angle of the lens in the original video always points to the tracking target. The panoramic camera extracts gyroscope data from the original video, and calculates the maximum acceleration and minimum acceleration during the movement according to the gyroscope data. The time when the maximum acceleration occurs is the starting point, and the time when the minimum acceleration occurs is the end of the movement. The middle moment between the starting point and the end of the movement is the highest point of the throw of the panoramic camera. The target video frame corresponding to the highest point of the throw is extracted from the adjusted original video, and the target video frame is repeatedly played, and the viewing angle of the target video frame is adjusted during the repeated playback so that the viewing angle of the target video frame rotates. Specifically, the terminal can be based on Figure 5 The method of the embodiment calculates the yaw angle and the pitch angle, and then rotates the viewing angle of the target video frame first to the upper left or upper right, and then to the lower left or lower right, until the viewing angle returns to the state of pointing to the tracking target.

[0139] In one embodiment, Figure 7 As shown, the video processing method includes the following steps:

[0140] S702, obtaining the original video and camera motion data collected by the camera during the movement process, and determining the first acceleration and the second acceleration of the camera during the movement process according to the camera motion data.

[0141] S704: Determine a target time when the camera reaches the target point during the movement according to a first time corresponding to the first acceleration and a second time corresponding to the second acceleration.

[0142] S706 , in response to the video acquisition instruction, identifying a tracking target in the acquired video, and acquiring tracking data corresponding to the tracking target.

[0143] S708, adjusting the viewing angle of the original video according to the tracking data, and extracting a target video frame corresponding to the target moment from the adjusted original video.

[0144] S710: Taking the target time as the rotation start time, and determining the rotation end time according to the rotation start time and a preset duration.

[0145] S712, determining the rotation period according to the rotation start time and the rotation end time, and calculating the ratio of each time point within the preset time length to the rotation period.

[0146] S714, based on the ratio and the initial yaw angle, calculating the yaw angle corresponding to each time point of the target video frame within a preset time length.

[0147] S716, based on the ratio and the initial pitch angle, calculating the pitch angle corresponding to each time point of the target video frame within a preset time length. The time point includes a first time point and a second time point.

[0148] S718, based on the yaw angle and the pitch angle corresponding to the first time point, the target video frame is rotated toward the upper left direction or the upper right direction, and the rotated target video frame at each first time point is copied to obtain a first video frame sequence.

[0149] S720: When the target video frame rotates to the first target position, based on the yaw angle and the pitch angle corresponding to the second time point, the target video frame is rotated toward the lower right direction or the lower left direction until it rotates to the second target position.

[0150] S722, copy the target video frame rotated at each second time point to obtain a second video frame sequence, and generate a rotated video based on the first video frame sequence and the second video frame sequence.

[0151] S724, inserting the rotated video into the original video according to the target time, obtaining the special effect video and playing it.

[0152] For the specific contents of S702 to S724 above, reference may be made to the specific implementation process above.

[0153] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0154] Based on the same inventive concept, the embodiment of the present application also provides a video processing device for implementing the above-mentioned video processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more video processing device embodiments provided below can refer to the limitations on the video processing method above, and will not be repeated here.

[0155] In one embodiment, Figure 8 As shown, a video processing device is provided, including: an acquisition module 802, a determination module 804, an extraction module 806, a calculation module 808 and a rotation processing module 810, wherein:

[0156] The acquisition module 802 is used to acquire the original video and camera motion data collected by the camera during the motion process;

[0157] A determination module 804 is used to determine a target time when the camera reaches a target point during the movement according to the camera movement data;

[0158] An extraction module 806 is used to extract a target video frame corresponding to a target moment from the original video;

[0159] The rotation processing module 808 is used to perform rotation processing on the target video frame according to the target time and the preset duration to obtain a rotated video.

[0160] In the above embodiment, the original video captured by the camera during the movement is obtained, and the target time when the camera reaches the target point during the movement is determined based on the camera motion data extracted from the original video. The target video frame corresponding to the target time is extracted from the original video. Therefore, the target video frame can be accurately determined based on the camera motion data, ensuring the accuracy of the determined target video frame. Then, according to the target time and the preset duration, the target video frame is rotated to obtain a rotated video. Since the computer device can accurately determine the target point reached by the camera during the movement based on the camera motion data, the efficiency of determining the target point is improved, and the visual effect of the rotated video is improved.

[0161] In one embodiment, the apparatus further comprises:

[0162] A calculation module 810 is used to calculate the yaw angle and pitch angle corresponding to each time point of the target video frame within the preset time length according to the target time and the preset time length;

[0163] The rotation processing module 808 is further used to perform rotation processing on the target video frame based on the yaw angle and the pitch angle to obtain a rotated video.

[0164] In one embodiment, the determination module 804 is further configured to:

[0165] Determine a first acceleration and a second acceleration of the camera during the movement according to the camera motion data;

[0166] A target time when the camera reaches the target point during the movement is determined according to a first time corresponding to the first acceleration and a second time corresponding to the second acceleration.

[0167] In one embodiment, the rotation processing module 808 is further configured to:

[0168] The target time is used as the rotation start time, and the rotation end time is determined according to the rotation start time and the preset duration;

[0169] Determine the rotation period according to the rotation start time and the rotation end time;

[0170] Calculate the ratio of each time point within a preset time length to the rotation period;

[0171] Based on the ratio and the initial yaw angle, the yaw angle corresponding to each time point of the target video frame within a preset duration is calculated;

[0172] Based on the ratio and the initial pitch angle, the pitch angle corresponding to each time point of the target video frame within a preset duration is calculated.

[0173] In one embodiment, each time point within the preset duration constitutes a time point set, and the time point set includes a first time point set and a second time point set; the first time point set consists of each first time point before the target video frame is rotated to the first target position; the second time point set consists of each second time point between the first target position and the second target position when the target video frame is rotated; the rotation processing module 808 is further used to:

[0174] Based on the yaw angle and the pitch angle corresponding to the first time point, the target video frame is rotated toward the upper left direction or the upper right direction;

[0175] Copying the target video frames rotated at each first time point to obtain a first video frame sequence;

[0176] When the target video frame rotates to the first target position, based on the yaw angle and the pitch angle corresponding to the second time point, the target video frame is rotated toward the lower right direction or the lower left direction until it rotates to the second target position;

[0177] Copying the target video frame rotated at each second time point to obtain a second video frame sequence;

[0178] A rotated video is generated based on the first video frame sequence and the second video frame sequence.

[0179] In one embodiment, the time point set composed of time points within the preset time length includes a third time point; the third time point is composed of each third time point in the process of causing the target video frame to rotate horizontally at the highest point of the movement process;

[0180] The rotation processing module 808 is also used to: when the target video frame rotates to the highest point, based on the yaw angle and pitch angle corresponding to the third time point, rotate the target video frame in the horizontal direction; when the target video frame rotates in the horizontal direction, the rotation speed component in the vertical direction is zero.

[0181] In one embodiment, Fig. 9 As shown, the device also includes:

[0182] An identification module 812, used to identify a tracking target in a pre-captured video;

[0183] The acquisition module 802 is also used to acquire the tracking data corresponding to the tracking target;

[0184] An adjustment module 814 is used to adjust the viewing angle of the original video according to the tracking data to obtain an adjusted original video;

[0185] The extraction module 806 is further configured to extract a target video frame corresponding to a target moment from the adjusted original video.

[0186] In one embodiment, the apparatus further comprises:

[0187] Insertion module 816, used for inserting the rotated video into the original video according to the target time to obtain a special effect video;

[0188] The playing module 818 is used to play the special effect video in response to a playing instruction for the special effect video.

[0189] Each module in the above video processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0190] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video processing method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0191] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0192] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0193] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0194] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0195] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0197] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video processing method, characterized in that: The method comprises: Obtain the original video and camera motion data collected by the camera during movement; Determining, based on the camera motion data, a target time at which the camera reaches a target point during the motion process; Extracting a target video frame corresponding to the target moment from the original video; According to the target time and the preset duration, the target video frame is rotated to obtain a rotated video.

2. The method according to claim 1, characterized in that: The step of rotating the target video frame according to the target time and the preset duration to obtain a rotated video comprises: According to the target time and the preset duration, calculating the yaw angle and the pitch angle corresponding to each time point of the target video frame within the preset duration; Based on the yaw angle and the pitch angle, the target video frame is rotated to obtain a rotated video.

3. The method according to claim 1, characterized in that Determining, according to the camera motion data, a target time at which the camera reaches a target point during the motion process includes: Determine, according to the camera motion data, a first acceleration and a second acceleration of the camera during the motion process; A target time when the camera reaches a target point during the movement is determined according to a first time corresponding to the first acceleration and a second time corresponding to the second acceleration.

4. The method according to claim 2, characterized in that: The calculating, according to the target time and the preset duration, the yaw angle and the pitch angle corresponding to each time point of the target video frame within the preset duration comprises: Taking the target time as the rotation start time, and determining the rotation end time according to the rotation start time and a preset duration; Determine the rotation period according to the rotation start time and the rotation end time; Calculating the ratio of each time point within the preset time length to the rotation period; Based on the ratio and the initial yaw angle, calculating the yaw angle of the target video frame corresponding to each time point within the preset duration; Based on the ratio and the initial pitch angle, the pitch angle of the target video frame corresponding to each time point within the preset duration is calculated.

5. The method according to claim 2, characterized in that: The time points within the preset duration constitute a time point set, and the time point set includes a first time point set and a second time point set; the first time point set consists of first time points before the target video frame is rotated to a first target position; the second time point set consists of second time points between the first target position and the second target position; the rotating process of the target video frame based on the yaw angle and the pitch angle to obtain a rotated video includes: Based on the yaw angle and the pitch angle corresponding to the first time point, rotating the target video frame toward the upper left direction or the upper right direction; Copying the target video frames rotated at each of the first time points to obtain a first video frame sequence; When the target video frame rotates to the first target position, based on the yaw angle and the pitch angle corresponding to the second time point, the target video frame is rotated toward the lower right direction or the lower left direction until it rotates to the second target position; Copying the target video frame rotated at each of the second time points to obtain a second video frame sequence; A rotated video is generated based on the first video frame sequence and the second video frame sequence.

6. The method according to claim 2, characterized in that The time points within the preset duration constitute a time point set; the time point set includes a third time point set; the third time point set is composed of each third time point in the process of causing the target video frame to rotate horizontally at the highest point of the movement process; the rotating process of the target video frame based on the yaw angle and the pitch angle to obtain the rotated video includes: When the target video frame rotates to the highest point, the target video frame is rotated in the horizontal direction based on the yaw angle corresponding to the third time point; when the target video frame rotates in the horizontal direction, the rotation speed component in the vertical direction is zero.

7. The method according to claim 1, characterized in that The method further comprises: Identify and track the target in the pre-captured video; Acquire tracking data corresponding to the tracking target; Adjusting the viewing angle of the original video according to the tracking data to obtain the adjusted original video; The step of extracting the target video frame corresponding to the target moment from the original video comprises: A target video frame corresponding to the target moment is extracted from the adjusted original video.

8. The method according to claim 1, characterized in that After rotating the target video frame according to the target time and the preset duration to obtain a rotated video, the method further includes: According to the target moment, inserting the rotated video into the original video to obtain a special effect video; In response to a play instruction for the special effects video, the special effects video is played.

9. The method according to any one of claims 1 to 8, characterized in that: The motion process is an upward throwing motion process; the target point is the highest point reached by the camera during the upward throwing motion process.

10. A video processing device, characterized in that: The device comprises: An acquisition module is used to acquire the original video and camera motion data collected by the camera during the movement; A determination module, configured to determine a target time when the camera reaches a target point during the movement according to the camera movement data; An extraction module, used for extracting a target video frame corresponding to the target moment from the original video; The rotation processing module is used to perform rotation processing on the target video frame according to the target time and the preset duration to obtain a rotated video.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images