Special effect video production method based on key frame interpolation, camera and storage medium

Abstract

The invention discloses a special effect video production method based on key frame interpolation, a camera and a storage medium, the method is implemented based on an improved Akima cubic Hermite interpolation algorithm, and the method comprises the following steps: S1, importing a spliced panoramic video into video editing software; s2, previewing the panoramic video in the S1 in an editing mode; s3, inserting or deleting a key frame; s4, selecting at least any one mode based on the key frames inserted or deleted in the step S3, and obtaining a parameter of a corresponding camera for shooting the panoramic video and recording the parameter as vi; s5, repeatedly inserting or deleting the key frame for a plurality of times; s6, performing spline polynomial interpolation for three times between the interpolation sample points (xi, vi) in the step S5 by using an improved Akima cubic Hermite interpolation algorithm; s7, processing the abrupt change area of the panoramic video in the mode selected in the step S4 by using the improved Akima cubic Hermite interpolation algorithm in the step S6; and S8, performing panoramic playing on the panoramic video and exporting the panoramic video. The method has the advantages of no need of manual intervention, time saving, labor saving and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing computer vision algorithms, and in particular relates to a special effects video production method based on key frame interpolation, a camera and a storage medium. Background Art

[0002] In the field of 360-degree sports cameras, the shooting equipment is usually composed of two dual fisheye cameras. The image data captured by the equipment is transmitted to the PC or mobile phone APP through wireless Wifi or Bluetooth or wired USB for editing and framing. According to the camera imaging principle and visual geometric relationship, general desktop PC video editing software or APP can only display the original image or video source originally shot, or more advanced ones can support gesture gyroscopes and other interactive viewing of various mode views, so that "crystal ball", "asteroid", "perspective view", "wide angle / ultra wide angle" and other types of views can be obtained. Although these modes are attractive to a certain extent, they still lack visual richness. Only one mode can be displayed at a time, which is too monotonous and requires frequent manual switching. In addition, when making special effects videos, there are sudden changes in the image during the mode switching process, the playback is unnatural, and the production efficiency is low. Summary of the invention

[0003] In order to solve the deficiencies in the prior art, the present invention provides a special effects video production method based on key frame interpolation with high production efficiency and natural playback, a camera and a storage medium.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] The method for producing special effects video based on key frame interpolation is characterized in that the method is implemented based on an improved Akimacubic Hermite interpolation algorithm and comprises the following steps:

[0006] S1. Import the stitched panoramic video into the video editing software;

[0007] S2, preview the panoramic video in S1 in the editing mode, and determine whether the panoramic video in the set time period meets the visual effects. If not, proceed to step S3; if yes, proceed to step S8;

[0008] S3, insert or delete a key frame, wherein the key frame defines the state matrix M of the camera shooting the panoramic video, and the mathematical expression of the state matrix M is:

[0009] M = PV, where P is the projection matrix and V is the view matrix;

[0010] S4, based on the key frames inserted or deleted in S3, select at least one mode, and obtain the corresponding parameters of the camera for shooting the panoramic video and record them as v i , where i represents the key frame selected for the i-th time;

[0011] S5. Repeat inserting or deleting key frames several times to obtain interpolation sample points (x i ,v i );

[0012] S6, using the improved Akima cubic Hermite interpolation algorithm to interpolate sample points (x i ,v i ) and calculate the slope using a modified average weighting, where if is the interval (x i ,x i+1 ), then at the sample point x i The differential weighted value d i The calculation is as follows:

[0013] The weight calculation formula is:

[0014]

[0015] S7, using the improved Akima cubic Hermite interpolation algorithm in S6 to process the mutation area of ​​the panoramic video in the mode selected in S4, to obtain a processed panoramic video;

[0016] S8. Play the panoramic video and export it.

[0017] Preferably, in step S3:

[0018]

[0019] Among them, θ is the field of view, a is the aspect ratio, n and f are the near and far distances;

[0020]

[0021] Among them, r is the right vector of the camera, which can be calculated by r = d × u, C = (C x , C y , C z ) is the camera position vector, each vector is a unit vector, × represents vector cross product, (d x , d y , d z ) is the camera orientation, (u x ,u y ,uz ) is the camera up vector.

[0022] Preferably, S7 also includes determining whether the special effects video meets the visual special effects requirements, if yes, proceeding to step S8, if no, proceeding to step S3.

[0023] The present invention also discloses a camera, which includes a processor and a memory. The processor is used to execute the steps of the special effect video production method based on key frame interpolation by calling the program or instruction stored in the memory.

[0024] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, and the program or instruction executes the steps of the above-mentioned special effect video production method based on key frame interpolation.

[0025] By adopting the above technical solution, the present invention has the following beneficial effects:

[0026] The present invention is based on the improved Akima cubic Hermite interpolation algorithm, and can smoothly transition in the selected mode, so as to process the sudden change area of ​​the panoramic video, so that the final panoramic video will not cause discomfort. In addition, the improved Akima cubic Hermite interpolation algorithm is implemented without manual intervention, which saves time and labor, has high production efficiency, and the panoramic video is played naturally.

[0027] In summary, the present invention has the advantages of no need for manual intervention, saving time and effort, high production efficiency, and natural playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments.

[0030] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention.

[0031] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0032] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0033] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] Example 1

[0035] In this embodiment, a special effects video production method, camera and storage medium based on key frame interpolation are proposed, which can solve the problems of lack of visual richness of special effects videos in the prior art. Most of the video editing software in the prior art has views such as "crystal ball", "asteroid", "perspective view", "wide angle / ultra wide angle", etc., but only displays one mode at a time, which is too monotonous and requires frequent manual switching. The video playback is too single, either does not support panoramic video, or only supports one mode. Even if there are multiple modes, manual switching or similar human intervention is required frequently. The video personalization setting is low, lacks certain visual appeal, and the mode switching is too simple. In the process, image mutations occur. If a complete video is to be played, the playback is extremely unnatural. In addition, it takes a lot of time and effort to edit a video with a certain attractive mode. The lack of automated process has low efficiency. The present invention can mainly solve the problem of automatic completion of special effect video production at the offline end (such as desktop PC and mobile phone APP) of video images taken by a 360-degree motion camera. According to a number of interpolation frames (key frames, each key frame has its own defined fixed mode, field of view angle fov, roll angle roll, pitch angle pitch, yaw angle yaw, distortion degree alpha, etc.) customized by the user in the panoramic video, a special effect video with a smooth transition (that is, a panoramic video, or a special 360-degree panoramic video) is automatically generated using the improved Akima cubic Hermite interpolation algorithm.

[0036] like Figure 1As shown, in one embodiment of the present invention, the special effects video production method based on key frame interpolation of the present invention is implemented based on the improved Akima cubic Hermite interpolation algorithm, and includes the following steps:

[0037] S1. Importing the stitched panoramic video into a video editing software. The video editing software may be an existing video editing software and may be used in conjunction with an improved Akima cubic Hermite interpolation algorithm. The video editing software includes various software for PC and mobile phones.

[0038] S2, preview the panoramic video in S1 in the editing mode, and determine whether the panoramic video in the set time period meets the visual effects. If not, proceed to step S3 (that is, the video editing user needs to insert a key frame under the custom frame). If yes, proceed to step S8, wherein the set time period in S2 can be set according to demand, such as 1S, 2S, etc.;

[0039] S3, insert or delete key frames. The selection of key frames can be specified arbitrarily in theory. However, if it is a video shot by a 360-degree motion camera, a certain action or scene that needs to be focused on is generally selected. The key frame defines the state matrix M of the camera shooting the panoramic video. The mathematical expression of the state matrix M is:

[0040] M = PV, where P is the projection matrix and V is the view matrix;

[0041] In step S3:

[0042]

[0043] Among them, θ is the field of view, a is the aspect ratio, n and f are the near and far distances, that is, n is the near distance and f is the far distance;

[0044]

[0045] Among them, r is the right vector of the camera, which can be calculated by r = d × u, C = (C x , C y , C z ) is the camera position vector, each vector is a unit vector, × represents vector cross product, (d x , d y , d z ) is the camera orientation, (u x ,u y ,u z ) is the camera up vector;

[0046] S4, based on the key frames inserted or deleted in S3, select at least one mode, such as two or more modes, and obtain the corresponding parameters of the camera for shooting the panoramic video and record them as v i , where i represents the key frame selected for the i-th time;

[0047] S5. Repeat inserting or deleting key frames several times to obtain interpolation sample points (x i ,v i ), that is, the video editing user can insert or delete multiple video key frames in the panoramic video multiple times, and finally obtain a series of interpolation sample points (x i ,v i );

[0048] S6. In order to obtain a video with smoother mode transition (smooth transition in one mode, smooth transition in two modes, or smooth transition in two modes), the interpolation sample points (x i ,v i ) and performs cubic spline polynomial interpolation between them, and is mathematically C1 continuous, and uses a modified average weighted weight to calculate the slope, where if is the interval (x i ,x i+1 ), then at the sample point x i The differential weighted value d i The calculation is as follows:

[0049] The weight calculation formula is:

[0050]

[0051] The weight calculation formula of the original Akima cubic Hermite interpolation algorithm is:

[0052] This formula gives equal weights on both sides of the sample point, which will average out fluctuations. In order to avoid this phenomenon, the weight formula is modified in the present invention. By modifying the weight formula, the improved Akimacubic Hermite interpolation algorithm will have less fluctuation disturbances and can more smoothly process the mutation area between modes (one or more), and visually the special effects video (panoramic video or even 360-degree panoramic video) will have a smoother transition;

[0053] S7, using the improved Akima cubic Hermite interpolation algorithm in S6 to process the mutation area of ​​the panoramic video in the mode selected in S4, to obtain a processed panoramic video, wherein S7 also includes determining whether the special effect video meets the visual special effect, if yes, proceeding to step S8, if not, proceeding to step S3, wherein the special effect video here is also the processed panoramic video;

[0054] S8, the panoramic video is played and exported, and the process ends.

[0055] The present invention mainly solves the problem of automatic generation of special effects video production in offline (desktop PC and mobile phone APP) of video images shot by 360 motion cameras. It can switch between various modes freely and automatically. Compared with the existing technology, its advantages are: improving the visual impact of the video, free transition playback of special effects video and automation of improved Akima cubic Hermite interpolation algorithm, improving efficiency. Traditional 360-degree panoramic videos either only support the original equirectangular type playback or a certain mode playback, and its mode is too monotonous and lacks certain appeal. The present invention can obviously switch freely in multiple modes without causing any visual discomfort, and it is natural and smooth. This is mainly due to the improvement of Akimacubic Hermite interpolation algorithm. Users only need to add "key frames" to the video frames of interest, and the entire panoramic video can automatically pass the "key frames". Without too much human intervention, a "blockbuster" special effects video can be generated! Thereby improving the efficiency of video editing, ensuring that novices in video editing can also get started quickly.

[0056] The present invention also discloses a camera, which includes a processor and a memory. The processor is used to execute the steps of the special effect video production method based on key frame interpolation by calling the program or instruction stored in the memory.

[0057] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, and the program or instruction executes the steps of the above-mentioned special effect video production method based on key frame interpolation.

[0058] This embodiment does not impose any formal limitation on the shape, material, structure, etc. of the present invention. Any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A special effects video production method based on key frame interpolation, characterized in that: The method is implemented based on the improved Akima cubic Hermite interpolation algorithm and includes the following steps: S1. Import the stitched panoramic video into the video editing software; S2, previewing the panoramic video in S1 in the editing mode, and determining whether the panoramic video in the set time period meets the visual effects, if not, proceeding to step S3, if yes, proceeding to step S8; S3, insert or delete a key frame, wherein the key frame defines the state matrix M of the camera shooting the panoramic video, and the mathematical expression of the state matrix M is: M = PV, where P is the projection matrix and V is the view matrix; S4, based on the key frames inserted or deleted in S3, select at least one mode, and obtain the corresponding parameters of the camera for shooting the panoramic video and record them as v i , where i represents the key frame selected for the i-th time; S5. Repeat inserting or deleting key frames several times to obtain interpolation sample points (x i ,v i ); S6, using the improved Akima cubic Hermite interpolation algorithm to interpolate sample points (x i ,v i ) and calculate the slope using a modified average weighting, where if is the interval (x i ,x i+1 ), then at the sample point x i The differential weighted value d i The calculation is as follows: The weight calculation formula is: S7, using the improved Akima cubic Hermite interpolation algorithm in S6 to process the mutation area of ​​the panoramic video in the mode selected in S4, to obtain a processed panoramic video; S8. Play the panoramic video and export it.

2. The special effects video production method based on key frame interpolation according to claim 1, characterized in that: In step S3: Among them, θ is the field of view, a is the aspect ratio, n and f are the near and far distances; Among them, r is the right vector of the camera, which can be calculated by r = d × u, C = (C x , C y , C z ) is the camera position vector, each vector is a unit vector, × represents vector cross product, (d x , d y , d z ) is the camera orientation, (u x ,u y ,u z ) is the camera up vector.

3. The special effects video production method based on key frame interpolation according to claim 1, characterized in that: The step S7 also includes determining whether the special effects video meets the visual special effects requirements. If yes, the process proceeds to step S8; otherwise, the process proceeds to step S3.

4. Camera, characterized in that: The camera includes a processor and a memory, and the processor is used to execute the steps of the special effects video production method based on key frame interpolation as described in any one of claims 1 to 3 by calling the program or instructions stored in the memory.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction executes the steps of the special effects video production method based on key frame interpolation as described in any one of claims 1 to 3.

Images