Video processing method, device, computer equipment and storage medium

Abstract

The present application provides a video processing method, device, computer equipment and storage medium, the method comprising determining the starting time point and the ending time point of the time period to be edited of the panoramic video and obtaining the corresponding Euler angles of the first key frame and the Euler angles of the second key frame, calculating the picture change amount of the first key frame and the second key frame, and determining the perspective transition mode between the first key frame and the second key frame according to the picture change amount, and then further determining the Euler angle of the transition image frame between the first key frame and the second key frame, and playing the planar video content of the time period to be edited according to the Euler angle of the transition image frame. The present application can automatically determine the perspective transition mode and the Euler angle of the transition image frame between two key frames, eliminating the tedious operation of the user manually selecting the perspective transition mode and manually adjusting the picture perspective, thereby improving the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and specifically to a video processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] Panoramic video is a video obtained by shooting 360 degrees in all directions with a panoramic camera. Existing panoramic cameras generally contain two or more fisheye lenses, and panoramic video is obtained by stitching multi-channel video data.

[0003] Since a panoramic video has a 360-degree viewing angle, users need to edit the video to form a flat video that matches their viewing angle when watching it. Existing methods for editing panoramic videos are usually done by keyframe dotting operations.

[0004] However, the keyframe dotting operation requires the user to manually rotate the spherical screen formed by the panoramic video continuously, and the user is also required to set the perspective transition mode between adjacent key frames. Therefore, the operation is very cumbersome, and the existing technical processing of transition image frames is not smooth enough. Therefore, there is an urgent need for a panoramic video processing method that can quickly generate a planar video without manual operation by the user. Summary of the invention

[0005] The embodiments of the present application provide a video processing method, apparatus, computer equipment and storage medium. The technical problem to be solved is that, in view of the defects of the prior art, in the existing panoramic video key frame processing process, the user needs to set the perspective transition mode between adjacent key frames and manually rotate the spherical screen formed by the panoramic video to generate a planar video, which is very cumbersome.

[0006] In order to solve the above technical problems, a technical solution adopted by the present application is: to provide a video processing method, apparatus, computer equipment and storage medium, wherein the video processing method comprises:

[0007] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0008] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0009] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0010] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0011] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0012] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0013] In a specific implementation, determining the perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame includes:

[0014] The picture change amount of the first key frame and the picture change amount of the second key frame are compared with the set picture change amount threshold C HT Make comparisons;

[0015] When C0 ≥ C HT , C1 ≥ C HT , determining that the perspective transition mode is a fast-in-fast-out mode;

[0016] When C0 <C HT , C1 <C HT , determining that the perspective transition mode is a slow-in-slow-out mode;

[0017] When C0 ≥ C HT , C1 <C HT , determining that the perspective transition mode is a fast-in-slow-out mode;

[0018] When C0 <C HT , C1 ≥ C HT , determine that the perspective transition mode is a slow-in-fast-out mode.

[0019] In a specific implementation, the posture information of the transition image frame between the first key frame and the second key frame is determined according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, and the posture information of the transition image frame is the Euler angle. ,include:

[0020] Said is a function of the Euler angles of the transition image frame with respect to the time point t, The curvature of changes smoothly during the time period to be edited, Monotonically increasing during the time period to be edited;

[0021] The Euler angles of the first keyframe Normalized to 0, that is To 0, the Euler angle of the second key frame Normalized to 1, that is is 1;

[0022] set up With control point P, the For the The normalized processing result of the control point P is , determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode;

[0023] According to the , and stated , to obtain the Functional relationship of

[0024] The normalized Euler angle of the transition image frame Perform inverse normalization to obtain the Euler angle of the transition image frame .

[0025] In a specific implementation, the determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode includes:

[0026] Will The coordinate system is rotated 45° counterclockwise to obtain a rotating coordinate system. The t' coordinate axis of the rotating coordinate system has a first intersection and a second intersection, the first intersection is the origin O, the second intersection is adjacent to the first intersection, and the control point P is located on the perpendicular bisector of the line connecting the first intersection and the second intersection;

[0027] The coordinates of the control point P in the rotating coordinate system are (s, h), where s is a fixed value and h is the distance from the control point P to the line connecting the first intersection point and the second intersection point;

[0028] The control point is in the Coordinates in the coordinate system , where R is the rotation matrix:

[0029] ;

[0030] in, The rotation angle is 45°;

[0031] According to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode, the size of h is determined, and according to the rotation matrix R, the control point P is determined in the Coordinates in the coordinate system .

[0032] In a specific implementation, determining the size of h according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode includes:

[0033] The picture change amount of the first key frame, the picture change amount of the second key frame and h satisfy:

[0034] h=HC0 C1, where H is a fixed value .

[0035] In a specific embodiment, the , and stated , to obtain the The functional relationship includes:

[0036] When the perspective transition mode is the fast-in-fast-out mode or the slow-in-slow-out mode, is a cubic function of the Euler angles of the transition image frame with respect to the time point t, The Euler angle of the transition intermediate image frame corresponding to the midpoint t3 of the time period to be edited symmetry;

[0037] The Euler angle of the transition intermediate image frame Normalized to 0.5, that is is 0.5;

[0038] Set the is a cubic equation of one variable: , then according to the , , and stated , to obtain the Functional relationship of

[0039] When the perspective transition mode is the fast-in-slow-out mode or the slow-in-fast-out mode, in the rotating coordinate system, Corresponding is a quadratic function of one variable and satisfies: ;

[0040] Set the is a quadratic equation: , as stated , And the rotation matrix R, we get is 0, is 0;

[0041] The coordinates of the control point P in the rotating coordinate system are: ;

[0042] According to the , and the coordinates of the control point P in the rotating coordinate system , to obtain the The functional relationship of , get the The functional relationship of .

[0043] In a specific implementation, the calculating the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame includes:

[0044] In a first adjacent time region of the first key frame, calculating the optical flow vectors of each feature point of each image frame in the first adjacent time region, normalizing the optical flow vectors of each feature point to obtain the optical flow vector modulus of each feature point, and averaging the optical flow vector modulus of each feature point to obtain the picture change amount of the first key frame;

[0045] In the second adjacent time region of the second key frame, the optical flow vectors of each feature point of each image frame in the second adjacent time region are calculated, and the optical flow vectors of each feature point are normalized to obtain the optical flow vector modulus of each feature point, and the optical flow vector modulus of each feature point are averaged to obtain the picture change amount of the second key frame.

[0046] In a specific implementation, a video processing device is also included, the device comprising:

[0047] A first determining module, the first determining module is used to determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0048] An acquisition module is used to acquire the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0049] A calculation module, the calculation module is used to calculate the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame;

[0050] a second determining module, the second determining module being used to determine a perspective transition mode between the start time point and the end time point according to a picture change amount of the first key frame and a picture change amount of the second key frame;

[0051] A third determining module is used to determine the posture information of the transition image frame between the first key frame and the second key frame according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, wherein the posture information of the transition image frame is the Euler angle. ;

[0052] A playing module is used to play the planar video content of the time period to be edited according to the Euler angles of the first key frame, the Euler angles of the second key frame and the Euler angles of the transition image frame.

[0053] In a specific implementation, a computer device is also included, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the video processing method described in any of the above specific implementations when executing the computer program.

[0054] In a specific implementation, it also includes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the video processing method described in any of the above specific implementations are implemented.

[0055] The beneficial effects of the present application are as follows: Different from the prior art, the embodiments of the present application provide a video processing method, apparatus, computer equipment and storage medium, wherein the video processing method includes:

[0056] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0057] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0058] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0059] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0060] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0061] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0062] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0063] The present application automatically determines the perspective transition mode between two key frames by calculating the picture change amount of the key frames, and calculates the perspective change process of the transition image frame based on the determined perspective transition mode and the picture change amount of the key frames, eliminating the tedious operations that require users to manually select the perspective transition mode and manually adjust the picture perspective, thereby improving the user experience.

[0064] This application improves the editing effect of panoramic videos, so that when editing panoramic videos, the user can automatically complete the smooth transition of perspectives between key frames marked by the user and adapt to the content of the picture, making the user's entire editing operation more efficient and concise. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 This is a schematic diagram of an implementation flow of a video processing method provided in Example 1 of the present application;

[0067] Figure 2 It is a perspective transition modeling diagram of a video processing method provided in Embodiment 1 of the present application when the perspective transition mode is a fast-in-fast-out mode or a slow-in-slow-out mode;

[0068] Figure 3 It is a perspective transition modeling diagram of a video processing method provided in Example 1 of the present application when the perspective transition mode is a fast-in-slow-out mode or a slow-in-fast-out mode;

[0069] Figure 4 It is a schematic diagram of the module composition of a video processing device provided in Embodiment 2 of the present application;

[0070] Figure 5 It is a schematic diagram of the internal structure of a computer device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0071] The present application is further described in detail below in conjunction with the accompanying drawings and implementation methods. It is particularly noted that the following implementation methods are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following implementation methods are only some implementation methods of the present application rather than all implementation methods. All other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0072] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to these processes, methods, products or devices.

[0073] The above terms are only for the convenience of description and should not be construed as limitations on the present technical solution.

[0074] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] The specific implementation of the present application is described in detail below in conjunction with specific embodiments.

[0076] Embodiment 1.

[0077] Figure 1 The implementation process of a video processing method provided in Example 1 of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown, which is described in detail as follows.

[0078] The starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video are determined.

[0079] It should be noted that the embodiments of the present application are applicable to video image processing devices such as video processing and acquisition, such as panoramic cameras or computer devices with video processing and display functions.

[0080] It should be noted that the panoramic video can be imported into the panoramic video editing software, and the panoramic video editing software can be deployed on the terminal device. After the panoramic video is imported into the panoramic video editing software, the user can click the play button on the panoramic video editing software to play the video screen. In the process of playing the video screen, if the user needs to edit the video screen of a certain period of time to be edited on the time progress bar, a starting point position and an ending point position can be selected on the time progress bar. The starting point position corresponds to the starting time point, and the ending point position corresponds to the ending time point. The time period between the starting time point and the ending time point is used as the time period to be edited. The video frame corresponding to the starting time point t1 is the first key frame, and the video frame corresponding to the ending time point t2 is the second key frame. The video frame of the time period to be edited between the first key frame and the second key frame is a transition image frame.

[0081] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle .

[0082] It should be noted that the panoramic video frame is equivalent to a panoramic spherical screen, and the user adjusting the viewing angle of the panoramic video frame is equivalent to rotating the panoramic spherical screen. Therefore, when the user performs the dot editing operation, it is equivalent to constantly adjusting the posture of the panoramic video frame. By obtaining the posture of the first key frame and the posture of the second key frame, and obtaining the posture information of the transition image frame through fitting calculation, the switching of the transition image frame can be completed, and a smooth transition from the first key frame to the second key frame can be achieved. The posture can be characterized and calculated by Euler angles.

[0083] According to the Euler angle of the first key frame and the Euler angles of the second keyframe Determine the posture information of the transition image frame between the first key frame and the second key frame, wherein the posture information of the transition image frame is the Euler angle .

[0084] The picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame are calculated.

[0085] It should be noted that the picture change amount of the first key frame is the picture change amount of the first key frame in the adjacent time area, that is, the picture change amount in the adjacent time area at the starting time point t1; the picture change amount of the second key frame is the picture change amount of the second key frame in the adjacent time area, that is, the picture change amount in the adjacent time area at the ending time point t2, and the adjacent time area includes the left adjacent time area, the right adjacent time area and the left and right adjacent time areas.

[0086] In one specific implementation manner, the calculating of the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame includes:

[0087] In a first adjacent time region of the first key frame, calculating the optical flow vectors of each feature point of each image frame in the first adjacent time region, normalizing the optical flow vectors of each feature point to obtain the optical flow vector modulus of each feature point, and averaging the optical flow vector modulus of each feature point to obtain the picture change amount of the first key frame;

[0088] In the second adjacent time region of the second key frame, the optical flow vectors of each feature point of each image frame in the second adjacent time region are calculated, and the optical flow vectors of each feature point are normalized to obtain the optical flow vector modulus of each feature point, and the optical flow vector modulus of each feature point are averaged to obtain the picture change amount of the second key frame.

[0089] It should be noted that the calculation of the optical flow vectors of each feature point of each image frame in the first adjacent time region may be the calculation of the feature points in the panoramic image frame of the panoramic video, or may be the calculation of the feature points only in the current viewing angle of the first key frame, so as to reduce the amount of calculation. Similarly, the calculation of the optical flow vectors of each feature point of each image frame in the second adjacent time region may be the calculation of the feature points in the panoramic image frame of the panoramic video, or may be the calculation of the feature points only in the current viewing angle of the second key frame.

[0090] The above-mentioned specific implementation method uses the optical flow method to calculate the optical flow vector of the feature point to realize the calculation of the picture change amount. Another specific implementation method may also be to realize the calculation of the picture change amount through target motion analysis, such as calculating the motion amplitude or motion state of the target object under the current viewing angle of the first key frame or the second key frame to calculate the current picture change amount.

[0091] A perspective transition mode between the start time point and the end time point is determined according to the picture change amount of the first key frame and the picture change amount of the second key frame.

[0092] It should be noted that by analyzing the picture change amount of the first key frame and the picture change amount of the second key frame, the drastic degree of picture change under the current picture perspective can be obtained, and then the appropriate perspective transition mode can be matched, which also presents a better picture transition perspective effect, and will not appear abrupt or disharmonious. There is no need for users to manually operate and select, and the transition effect of the picture can be grasped more accurately.

[0093] One specific implementation method is that determining the perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame includes:

[0094] The picture change amount of the first key frame and the picture change amount of the second key frame are compared with the set picture change amount threshold C HT Make comparisons;

[0095] When C0 ≥ C HT , C1 ≥ C HT , determining that the perspective transition mode is a fast-in-fast-out mode;

[0096] When C0 <C HT , C1 <C HT , determining that the perspective transition mode is a slow-in-slow-out mode;

[0097] When C0 ≥ C HT , C1 <C HT , determining that the perspective transition mode is a fast-in-slow-out mode;

[0098] When C0 <C HT , C1 ≥ C HT , determine that the perspective transition mode is a slow-in-fast-out mode.

[0099] It should be noted that the picture change amount of the first key frame and the picture change amount of the second key frame are compared with the set picture change amount threshold C HT By comparing, the drastic degree of the picture change between the first key frame and the second key frame can be obtained. When C0≥C HT , C1 ≥ C HT When C0 is 0, it means that the picture changes between the first key frame and the second key frame are more drastic and the picture changes are larger. In order to make the picture smoother, a fast-in-fast-out perspective transition mode is selected between the first key frame and the second key frame to obtain a better picture transition effect. <C HT , C1 <C HTWhen C0≥C1, it means that the picture changes between the first key frame and the second key frame are relatively smooth and the picture changes are not large. In order to make the picture transition smooth, a slow-in and slow-out perspective transition mode is selected between the first key frame and the second key frame to obtain a better picture transition effect. HT , C1 <C HT Similarly, when the first key frame has a large picture change, the fast-in perspective transition mode is selected, and when the second key frame has a small picture change, the slow-out perspective transition mode is selected to adapt to the change range of the picture, obtain a better perspective transition effect, and ensure the smoothness of the picture; when the C0 <C HT , C1 ≥ C HT Similarly, in the first key frame where the picture changes little, the slow-in perspective transition mode is selected, and in the second key frame where the picture changes greatly, the fast-out perspective transition mode is selected to adapt to the change range of the picture, obtain better perspective transition effect, and ensure the smoothness of the picture.

[0100] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle .

[0101] It should be noted that, when the perspective transition mode is determined, the posture information of each transition image frame between the first key frame and the second key frame can be further determined.

[0102] One specific implementation method is that the posture information of the transition image frame between the first key frame and the second key frame is determined according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, and the posture information of the transition image frame is the Euler angle ,include:

[0103] Said is a function of the Euler angles of the transition image frame with respect to the time point t, The curvature of changes smoothly during the time period to be edited, Monotonically increasing during the time period to be edited;

[0104] The Euler angles of the first keyframe Normalized to 0, that is To 0, the Euler angle of the second key frame Normalized to 1, that is is 1;

[0105] set up With control point P, the For the The normalized processing result of the control point P is , determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode;

[0106] According to the , and stated , to obtain the Functional relationship of

[0107] The normalized Euler angle of the transition image frame Perform inverse normalization to obtain the Euler angle of the transition image frame .

[0108] It should be noted that by changing the Euler angle of the first key frame The Euler angles of the first key frame are normalized to simplify the calculation process and improve the calculation efficiency. Normalized to 0, that is To 0, the Euler angle of the second key frame Normalized to 1, that is is 1, such as Figure 2 As shown, in In the coordinate system, For the The normalized result of the processing is that the origin O corresponds to the , the coordinate (1, 1) corresponds to the .

[0109] Said There is also a control point P, the coordinates of which are , the control point P controls the The curvature change amplitude reflects the speed of the picture perspective transition at the first key frame and the second key frame. Therefore, the coordinates of the control point P are determined according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode.

[0110] Finally, according to the , and stated , we can obtain the The functional relationship of the normalized Euler angle of the transition image frame is Perform inverse normalization to obtain the Euler angle of the transition image frame .

[0111] One specific implementation manner is that the normalized Euler angles of the transition image frame are denormalized to obtain the Euler angles of the transition image frame, specifically:

[0112] .

[0113] One specific implementation manner is that determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode includes:

[0114] Will The coordinate system is rotated 45° counterclockwise to obtain a rotating coordinate system. The t' coordinate axis of the rotating coordinate system has a first intersection and a second intersection, the first intersection is the origin O, the second intersection is adjacent to the first intersection, and the control point P is located on the perpendicular bisector of the line connecting the first intersection and the second intersection;

[0115] The coordinates of the control point P in the rotating coordinate system are (s, h), where s is a fixed value and h is the distance from the control point P to the line connecting the first intersection point and the second intersection point;

[0116] The control point is in the Coordinates in the coordinate system , where R is the rotation matrix:

[0117] ;

[0118] in, The rotation angle is 45°;

[0119] According to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode, the size of h is determined, and according to the rotation matrix R, the control point P is determined in the Coordinates in the coordinate system .

[0120] It should be noted that the rotating coordinate system is The coordinate system is rotated 45° counterclockwise, so the rotation matrix R is:

[0121] ;

[0122] in, The rotation angle is 45°;

[0123] like Figure 2 and Figure 3 As shown, the rotating coordinate system is , in the rotating coordinate system In With the rotating coordinate system The t' coordinate axis has a first intersection and a second intersection, the first intersection is the origin O, the second intersection is M, the first intersection O is adjacent to the second intersection M, the control point P is located on the perpendicular bisector of OM, the distance from the control point P to OM is h, and the geometric meaning of h is the distance that the control point P deviates from the symmetry, that is, it reflects the The degree of change at the first key frame and the second key frame, so h is directly related to the picture change amount of the first key frame and the picture change amount of the second key frame.

[0124] According to the rotation matrix R, the control point P is Coordinates in the coordinate system Therefore, the size of h can be determined according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode. Coordinate system and the rotation coordinate system The position relationship of h can be determined to determine the positive or negative of h, that is, to determine the position of the control point P in the Coordinates in the coordinate system , it is also possible to determine the control point P in the rotating coordinate system The coordinates (s, h) in .

[0125] One specific implementation method is that determining the size of h according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode includes:

[0126] The picture change amount of the first key frame, the picture change amount of the second key frame and h satisfy: h=HC0 C1, where H is a fixed value .

[0127] It should be noted that h is positively correlated with the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame. H reflects the value space of h. The value space of h reflects the value range of h, and the value range is related to the perspective transition mode.

[0128] One specific implementation method is that according to the , and stated , to obtain the The functional relationship includes:

[0129] When the perspective transition mode is the fast-in-fast-out mode or the slow-in-slow-out mode, is a cubic function of the Euler angles of the transition image frame with respect to the time point t, The Euler angle of the transition intermediate image frame corresponding to the midpoint t3 of the time period to be edited symmetry;

[0130] The Euler angle of the transition intermediate image frame Normalized to 0.5, that is is 0.5;

[0131] Set the is a cubic equation of one variable: , then according to the , , and stated , to obtain the Functional relationship of

[0132] When the perspective transition mode is the fast-in-slow-out mode or the slow-in-fast-out mode, in the rotating coordinate system, Corresponding is a quadratic function that satisfies: ;

[0133] Set the is a quadratic equation: , as stated , And the rotation matrix R, we get is 0, is 0;

[0134] The coordinates of the control point P in the rotating coordinate system are: ;

[0135] According to the , and the coordinates of the control point P in the rotating coordinate system , to obtain the The functional relationship of , get the The functional relationship of .

[0136] It should be noted that when the perspective transition mode is the fast-in-fast-out mode or the slow-in-slow-out mode, the is a cubic function of the Euler angles of the transition image frame with respect to the time point t, The Euler angle of the transition intermediate image frame corresponding to the midpoint t3 of the time period to be edited Symmetrical, such as Figure 2 As shown, Figure 2The response is in the fast-in-fast-out mode or the slow-in-slow-out mode. The process changes with time t.

[0137] Set the is a cubic equation of one variable: , then according to the , , and stated , to obtain the Functional relationship:

[0138] according to =0, =1, = 0.5, , establish the The vector representation of the linear equations is:

[0139] ;

[0140] And get the matrix equation: Ax=B;

[0141] From the solution of the matrix equation, it can be seen that when the rank of the augmented matrix is ​​equal to the rank of the coefficient matrix, that is, R(A)=R(A,B), the linear equation system has a unique solution. Therefore, the above can be obtained by calculation. The solution.

[0142] Among them, it can be obtained from the geometric relationship that the control point P in the rotating coordinate system The coordinates below are , the control point P is in the Coordinates in the coordinate system , find the control point P in the Coordinates in the coordinate system , and then obtain the The solution.

[0143] Furthermore, from the geometric relationship, the value range of h is: ,in:

[0144] When h=0, since the control point P and OM are on the same line, The curve equation degenerates into a straight line, which corresponds to the linear perspective transition mode;

[0145] when When the first key frame just begins The slope of the curve is greater than 1, and when approaching the second key frame The slope of the curve is also greater than 1, which corresponds to the fast-in-fast-out perspective transition mode;

[0146] when When the first key frame just begins The slope of the curve is less than 1, and when approaching the second key frame The slope of the curve is also less than 1, which corresponds to a slow-in and slow-out perspective transition mode.

[0147] like Figure 3 As shown, when the perspective transition mode is the fast-in-slow-out mode or the slow-in-fast-out mode, in the In the coordinate system, The control point P is symmetrically distributed in the rotating coordinate system. Next, the Corresponding is a quadratic function of one variable and satisfies: ;

[0148] Set the is a quadratic equation: , as stated , And the rotation matrix R, we get is 0, is 0;

[0149] From the geometric relationship, it can be obtained that the control point P in the rotating coordinate system The coordinates below are ;

[0150] According to the , and the control point P in the rotated coordinate system The coordinates below , to obtain the The functional relationship is: , and according to , get the The functional relationship of .

[0151] According to the geometric relationship constraints, , and due to the In the The coordinate system is monotonically increasing, that is, a certain perspective will not appear repeatedly at multiple time points in the time period to be edited, that is, it will not oscillate multiple times at a certain perspective position during the time transition of the video. Therefore, due to the characteristics of the univariate quadratic curve, in the rotating coordinate system Below, with: .

[0152] Combined with the above The curve equation can be obtained as follows: ;

[0153] in:

[0154] When h=0, since the control point P and OM are on the same line, The curve equation degenerates into a straight line, which corresponds to the linear perspective transition mode;

[0155] when When the first key frame just begins The slope of the curve is greater than 1, and when approaching the second key frame The slope of the curve is less than 1, which corresponds to the fast-in and slow-out perspective transition mode;

[0156] when When the first key frame just begins The slope of the curve is less than 1, and when approaching the second key frame The slope of the curve is greater than 1, which corresponds to a slow-in-fast-out perspective transition mode.

[0157] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0158] It should be noted that the posture information of each transition image frame can be obtained according to the Euler angles of the transition image frame, thereby realizing the Euler angles of each transition image frame from the first key frame to the second key frame, so that each frame can be accurately and smoothly transitioned according to the Euler angles of each transition image frame, thereby achieving a smooth transition effect of key frame dotting of the panoramic video.

[0159] Embodiment 2.

[0160] Figure 4 A schematic diagram of the module composition of a video processing device provided in the second embodiment of the present application is shown. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0161] A video processing device according to an embodiment of the present application includes:

[0162] A first determining module, the first determining module is used to determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0163] An acquisition module is used to acquire the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0164] A calculation module, the calculation module is used to calculate the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame;

[0165] a second determining module, the second determining module being used to determine a perspective transition mode between the start time point and the end time point according to a picture change amount of the first key frame and a picture change amount of the second key frame;

[0166] A third determining module is used to determine the posture information of the transition image frame between the first key frame and the second key frame according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, wherein the posture information of the transition image frame is the Euler angle. ;

[0167] A playing module is used to play the planar video content of the time period to be edited according to the Euler angles of the first key frame, the Euler angles of the second key frame and the Euler angles of the transition image frame.

[0168] A video processing device according to an embodiment of the present application is used to implement a video processing method provided in any specific implementation manner of the first embodiment of the present application:

[0169] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0170] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0171] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0172] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0173] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0174] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0175] It should be noted that each module in the above video processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0176] It should be noted that for technical details that are not described in detail in this embodiment, reference can be made to the video processing method provided in the above-mentioned embodiment 1, and will not be repeated here.

[0177] Embodiment three.

[0178] In an embodiment of the present application, a computer device is provided. Figure 5 A schematic diagram of the internal structure of a computer device.

[0179] The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the video processing method in any specific implementation manner provided in Embodiment 1 of the present application, specifically including:

[0180] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0181] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0182] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0183] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0184] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0185] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0186] It should be noted that the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video generation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0187] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0188] It should be noted that for technical details that are not described in detail in this embodiment, reference can be made to the video processing method provided in the above-mentioned embodiment 1, and will not be repeated here.

[0189] Embodiment 4.

[0190] In an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the video processing method in any specific implementation manner provided in the first embodiment of the present application is implemented, specifically including:

[0191] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0192] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0193] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0194] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0195] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0196] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0197] It should be noted that for technical details that are not described in detail in this embodiment, reference can be made to the video processing method provided in the above-mentioned embodiment 1, and will not be repeated here.

[0198] Different from the prior art, the embodiments of the present application provide a video processing method, apparatus, computer device and storage medium, wherein the video processing method includes:

[0199] Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video;

[0200] Obtain the posture information of the first key frame corresponding to the starting time point and the posture information of the second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle , the posture information of the second key frame is the Euler angle ;

[0201] Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame;

[0202] Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame;

[0203] According to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame is determined, and the posture information of the transition image frame is the Euler angle ;

[0204] The planar video content of the time period to be edited is played according to the Euler angles of the first key frame, the Euler angles of the second key frame, and the Euler angles of the transition image frame.

[0205] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0206] The present application automatically determines the perspective transition mode between two key frames by calculating the picture change amount of the key frames, and calculates the perspective change process of the transition image frame based on the determined perspective transition mode and the picture change amount of the key frames, eliminating the tedious operations that require users to manually select the perspective transition mode and manually adjust the picture perspective, thereby improving the user experience.

[0207] This application improves the editing effect of panoramic videos, so that when editing panoramic videos, the user can automatically complete the smooth transition of perspectives between key frames marked by the user and adapt to the content of the picture, making the user's entire editing operation more efficient and concise.

[0208] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0210] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a terminal device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0211] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video processing method, characterized in that: The video processing method comprises: Determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video; Acquire posture information of a first key frame corresponding to the starting time point and posture information of a second key frame corresponding to the ending time point, wherein the posture information of the first key frame is the Euler angle φ(t1), and the posture information of the second key frame is the Euler angle φ(t2); Calculating a picture change amount C0 of the first key frame and a picture change amount C1 of the second key frame; Determining a perspective transition mode between the start time point and the end time point according to the picture change amount of the first key frame and the picture change amount of the second key frame; Determine, according to the Euler angles of the first key frame, the Euler angles of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame, wherein the posture information of the transition image frame is the Euler angle φ(t); Playing the planar video content of the time period to be edited according to the Euler angle of the first key frame, the Euler angle of the second key frame and the Euler angle of the transition image frame; The determining, according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame, wherein the posture information of the transition image frame is the Euler angle φ(t), comprises: The φ(t) is a function of the Euler angle of the transition image frame with respect to the time point t, the curvature of the φ(t) changes smoothly in the time period to be edited, and the φ(t) increases monotonically in the time period to be edited; The Euler angle φ(t1) of the first key frame is normalized to 0, that is, φ - (0) is 0, and the Euler angle φ(t2) of the second key frame is normalized to 1, that is, φ - (1) is 1; Setting φ - (t) has a control point P, the φ - (t) is the normalized result of φ(t), and the coordinates of the control point P are (t c ,φ - (t c )), determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode; According to the - (0) - (1) and the φ - (t c ), and obtain the φ - (t) The normalized Euler angle φ of the transition image frame - (t) is denormalized to obtain the Euler angle φ(t) of the transition image frame.

2. A video processing method according to claim 1, characterized in that: The determining, according to the picture change amount of the first key frame and the picture change amount of the second key frame, a perspective transition mode between the start time point and the end time point comprises: The picture change amount of the first key frame and the picture change amount of the second key frame are compared with the set picture change amount threshold C HT Make comparisons; When C0 ≥ C HT , C1 ≥ C HT , determining that the perspective transition mode is a fast-in-fast-out mode; When C0 <C HT , C1 <C HT , determining that the perspective transition mode is a slow-in-slow-out mode; When C0 ≥ C HT , C1 <C HT , determining that the perspective transition mode is a fast-in-slow-out mode; When C0 <C HT , C1 ≥ C HT , determine that the perspective transition mode is a slow-in-fast-out mode.

3. A video processing method according to claim 2, characterized in that: The determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode includes: φ - The (t)-t coordinate system is rotated 45° counterclockwise to obtain a rotating coordinate system. - (t) has a first intersection and a second intersection with the t' coordinate axis of the rotating coordinate system, the first intersection is the origin O, the second intersection is adjacent to the first intersection, and the control point P is located on the perpendicular bisector of the line connecting the first intersection and the second intersection; The coordinates of the control point P in the rotating coordinate system are (s, h), where s is a fixed value and h is the distance from the control point P to the line connecting the first intersection point and the second intersection point; The control point is in the - The coordinates in the (t)-t coordinate system (t c ,φ - (t c ))=(s,h)R, where R is the rotation matrix: Where θ is the rotation angle of 45°; According to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode, the size of h is determined, and according to the rotation matrix R, the control point P is determined in the φ - The coordinates in the (t)-t coordinate system (t c ,φ - (t c )).

4. A video processing method according to claim 3, characterized in that: The determining the size of h according to the picture change amount of the first key frame, the picture change amount of the second key frame and the view transition mode includes: The picture change amount of the first key frame, the picture change amount of the second key frame and h satisfy: h=HC0 C1, where H is a fixed value 5. A video processing method according to claim 4, characterized in that: According to the - (0) - (1) and the φ - (t c ), and obtain the φ - (t), including: When the perspective transition mode is the fast-in-fast-out mode or the slow-in-slow-out mode, the φ - (t) is a cubic function of the Euler angle of the transition image frame with respect to the time point t, and φ - The Euler angle φ of the transitional intermediate image frame corresponding to the midpoint t3 of the time period to be edited is - (t3) symmetry; The Euler angle φ of the transition intermediate image frame - (t3) is normalized to 0.5, that is, φ - (0.5) is 0.5; Set the φ - (t) is a cubic equation of one variable: φ - (t) = a + bt + ct 2 +dt 3 , then according to the φ - (0) - (1) The φ - (0.5) and the φ - (t c ), and obtain the φ - (t) When the perspective transition mode is the fast-in-slow-out mode or the slow-in-fast-out mode, in the rotating coordinate system, the φ - (t) corresponding to φ - R (t') is a quadratic function of one variable and satisfies: φ - (t) = φ - R (t')R; Set the φ - R (t') is a quadratic equation: φ - R (t') = m + nt + pt 2 , by the φ - (0) - (1) and the rotation matrix R, we get φ - R (0) is 0, is 0; The coordinates of the control point P in the rotating coordinate system are: According to the above φ - R (0) and the coordinates of the control point P in the rotating coordinate system Obtain the φ - R (t), and according to φ - (t) = φ - R (t)R, we get the φ - (t) is a function of 6. A video processing method according to any one of claims 1 to 5, characterized in that: The calculating the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame includes: In a first adjacent time region of the first key frame, calculating the optical flow vectors of each feature point of each image frame in the first adjacent time region, normalizing the optical flow vectors of each feature point to obtain the optical flow vector modulus of each feature point, and averaging the optical flow vector modulus of each feature point to obtain the picture change amount of the first key frame; In the second adjacent time region of the second key frame, the optical flow vectors of each feature point of each image frame in the second adjacent time region are calculated, and the optical flow vectors of each feature point are normalized to obtain the optical flow vector modulus of each feature point, and the optical flow vector modulus of each feature point are averaged to obtain the picture change amount of the second key frame.

7. A video processing device, characterized in that: The device comprises: A first determining module, the first determining module is used to determine the starting time point t1 and the ending time point t2 of the time period to be edited of the panoramic video; An acquisition module, the acquisition module is used to acquire posture information of a first key frame corresponding to the starting time point and posture information of a second key frame corresponding to the ending time point, the posture information of the first key frame is the Euler angle φ(t1), and the posture information of the second key frame is the Euler angle φ(t2); A calculation module, the calculation module is used to calculate the picture change amount C0 of the first key frame and the picture change amount C1 of the second key frame; a second determining module, the second determining module being used to determine a perspective transition mode between the start time point and the end time point according to a picture change amount of the first key frame and a picture change amount of the second key frame; a third determining module, the third determining module being used to determine the posture information of the transition image frame between the first key frame and the second key frame according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, wherein the posture information of the transition image frame is the Euler angle φ(t); The determining, according to the Euler angle of the first key frame, the Euler angle of the second key frame and the perspective transition mode, the posture information of the transition image frame between the first key frame and the second key frame, wherein the posture information of the transition image frame is the Euler angle φ(t), comprises: The φ(t) is a function of the Euler angle of the transition image frame with respect to the time point t, the curvature of the φ(t) changes smoothly in the time period to be edited, and the φ(t) increases monotonically in the time period to be edited; The Euler angle φ(t1) of the first key frame is normalized to 0, that is, φ - (0) is 0, and the Euler angle φ(t2) of the second key frame is normalized to 1, that is, φ - (1) is 1; Setting φ - (t) has a control point P, the φ - (t) is the normalized result of φ(t), and the coordinates of the control point P are (t c ,φ - (t c )), determining the coordinates of the control point P according to the picture change amount of the first key frame, the picture change amount of the second key frame and the perspective transition mode; According to the - (0) - (1) and the φ - (t c ), and obtain the φ - (t) The normalized Euler angle φ of the transition image frame - (t) performing inverse normalization to obtain the Euler angle φ(t) of the transition image frame; A playing module is used to play the planar video content of the time period to be edited according to the Euler angles of the first key frame, the Euler angles of the second key frame and the Euler angles of the transition image frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the video processing method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video processing method according to any one of claims 1 to 6 are implemented.

Images