Video generation method and device, computer device and storage medium

Abstract

The application relates to a video generation method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring images obtained by surrounding shooting of a target object by a shooting device, determining first centripetal acceleration and second centripetal acceleration corresponding to the shooting of each image by the shooting device; the first centripetal acceleration and the second centripetal acceleration are centripetal accelerations of the shooting device in different coordinate systems; a rotation axis of the shooting device in a world coordinate system is calculated according to the first centripetal acceleration; a space rotation amount of the shooting device in the world coordinate system is calculated according to the rotation axis and the second centripetal acceleration; each image is spatially converted according to each space rotation amount, and each clipping perspective image is obtained; and each clipping perspective image is combined to obtain a target video. The method can automatically identify a target object in surrounding shooting, thereby efficiently generating a target video shot around the target object, and the target video can realize a slow-motion special effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a video generation method and device, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] With the development of image processing technology, more and more dynamic images are taken, which include the process of a shooting device taking pictures around a target object. The images taken by the shooting device around the target object can be the process of the shooting device moving, or the process of the entire scene moving around.

[0003] When generating a target video based on these images, a professional person needs to select key frames in the images and set a rotation amount for each key frame by using a professional editing tool, which is a large amount of work and low in efficiency. SUMMARY

[0004] Therefore, it is necessary to provide a video generation method, device, computer equipment, computer readable storage medium and computer program product capable of efficiently surrounding a target object to solve the above technical problems.

[0005] In a first aspect, the present application provides a video generation method. The method comprises:

[0006] obtaining images taken by a shooting device around a target object, and determining a first centripetal acceleration and a second centripetal acceleration corresponding to the shooting device when taking each of the images; the first centripetal acceleration and the second centripetal acceleration are centripetal accelerations of the shooting device in different coordinate systems;

[0007] calculating a rotation axis of the shooting device in a world coordinate system according to the first centripetal acceleration;

[0008] calculating a spatial rotation amount of the shooting device in the world coordinate system according to the rotation axis and the second centripetal acceleration;

[0009] spatially converting each of the images according to each of the spatial rotation amounts to obtain each of the edited perspective images;

[0010] combining each of the edited perspective images to obtain a target video.

[0011] In one of the embodiments, the determining of the first centripetal acceleration and the second centripetal acceleration corresponding to the shooting device when taking each of the images comprises:

[0012] determining an acceleration of the shooting device in a camera coordinate system when taking pictures around the target object;

[0013] The acceleration of the shooting device in the world coordinate system is determined based on the rotation matrix and the acceleration of the shooting device in the camera coordinate system.

[0014] The acceleration components of the shooting device in the world coordinate system are extracted to obtain the first centripetal acceleration and the second centripetal acceleration of the shooting device.

[0015] In one embodiment, the step of extracting components of the acceleration of the imaging device in the world coordinate system to obtain the first centripetal acceleration and the second centripetal acceleration of the imaging device includes:

[0016] The first centripetal acceleration is determined based on the acceleration of the shooting device in the world coordinate system and the acceleration due to gravity.

[0017] The fusion result between the first centripetal acceleration and the rotation matrix is ​​filtered by a low-pass filter to obtain the second centripetal acceleration.

[0018] Based on the rotation matrix and the second centripetal acceleration, the first centripetal acceleration is denoised to obtain the denoised first centripetal acceleration;

[0019] The step of calculating the rotation axis of the imaging device in the world coordinate system according to the first centripetal acceleration includes:

[0020] The rotation axis of the shooting device in the world coordinate system is calculated based on the first centripetal acceleration after noise reduction.

[0021] In one embodiment, the step of extracting components of the acceleration of the imaging device in the world coordinate system to obtain the first centripetal acceleration and the second centripetal acceleration of the imaging device includes:

[0022] The acceleration of the imaging device in the world coordinate system is high-pass filtered to obtain the first centripetal acceleration;

[0023] The second centripetal acceleration is obtained based on the rotation matrix and the first centripetal acceleration.

[0024] In one embodiment, the method further includes:

[0025] When the imaging device captures each of the images, the timestamp and angular velocity corresponding to each of the images are obtained;

[0026] Based on the acceleration and angular velocity of the shooting device in the camera coordinate system, the rotation between the camera coordinate system and the world coordinate system is generated.

[0027] The rotation matrix is ​​generated based on the rotation between the camera coordinate system and the world coordinate system.

[0028] In one of the embodiments, the generating the rotation matrix based on the rotation amount between the camera coordinate system and the world coordinate system comprises: smoothing the rotation amount to obtain the rotation matrix when a timestamp of the photographing device is synchronized with a timestamp corresponding to each of the images.

[0029] In one of the embodiments, the calculating the rotation axis of the photographing device in the world coordinate system according to the first centripetal acceleration comprises:

[0030] According to the candidate rotation axis of the photographing device in the world coordinate system and the first centripetal acceleration corresponding to each of the images, a combination value of the candidate rotation axis is calculated.

[0031] According to the combination value, a target rotation axis of the photographing device in the world coordinate system is determined from the candidate rotation axis.

[0032] The calculating the spatial rotation amount of the photographing device in the world coordinate system according to the rotation axis and the second centripetal acceleration comprises:

[0033] The calculating the spatial rotation amount of the photographing device in the world coordinate system according to the target rotation axis and the second centripetal acceleration.

[0034] In one of the embodiments, the spatial rotation amount comprises first, second and third vectors with different directions; the calculating the spatial rotation amount of the photographing device in the world coordinate system according to the rotation axis and the second centripetal acceleration comprises:

[0035] The first vector is generated according to the second centripetal acceleration.

[0036] According to an included angle between the target rotation axis and the direction of gravity, it is determined whether to generate the second vector based on the rotation matrix.

[0037] If yes, the second vector is generated based on the first vector, the target rotation axis and the rotation matrix, and the first vector and the second vector are combined to obtain the third vector.

[0038] If no, the second vector is generated based on the first vector and a preset matrix, and the first vector and the second vector are combined to obtain the third vector.

[0039] In one of the embodiments, the spatial rotation amount comprises first and second vectors with different directions; the clipping perspective image comprises first and second clipping perspective images; and the combining each of the clipping perspective images comprises:

[0040] calculate a first angle based on the first vector and the second vector of the first clip perspective image; the first angle is used to represent a clip time corresponding to the first clip perspective image;

[0041] calculate a second angle based on the first vector and the second vector of the second clip perspective image; the second angle is used to represent a clip time corresponding to the second clip perspective image;

[0042] calculate an angle difference between the first angle and the second angle;

[0043] combine the first clip perspective image and the second clip perspective image according to the angle difference.

[0044] In one of the embodiments, the spatial conversion of each image according to the spatial rotation amount to obtain each clip perspective image comprises:

[0045] determine a mapping relationship between each photographed image and a panoramic perspective planar image;

[0046] map each image based on the mapping relationship to obtain a panoramic perspective planar image corresponding to each image respectively;

[0047] perform color interpolation resampling on each panoramic perspective planar image to obtain each clip perspective image.

[0048] In a second aspect, the application further provides a video generation device. The device comprises:

[0049] a data acquisition module configured to acquire images obtained by a shooting device surrounding a shooting target object, and determine a first centripetal acceleration and a second centripetal acceleration corresponding to the shooting of each image by the shooting device; the first centripetal acceleration and the second centripetal acceleration are centripetal accelerations of the shooting device in different coordinate systems;

[0050] a rotation axis calculation module configured to calculate a rotation axis of the shooting device in a world coordinate system according to the first centripetal acceleration;

[0051] a spatial rotation calculation module configured to calculate a spatial rotation amount of the shooting device in the world coordinate system according to the rotation axis and the second centripetal acceleration;

[0052] an image conversion module configured to perform spatial conversion on each image according to each spatial rotation amount to obtain each clip perspective image;

[0053] a target video generation module configured to combine each clip perspective image to obtain a target video.

[0054] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the video generation method in any of the above embodiments when executing the computer program.

[0055] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program, when executed by a processor, implements the steps of the video generation method in any of the above embodiments.

[0056] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program, when executed by a processor, implements the steps of the video generation method in any of the above embodiments.

[0057] The video generation method, device, computer device, storage medium and computer program product described above obtain images obtained by a shooting device surrounding a target object, determine first centripetal acceleration and second centripetal acceleration corresponding to the shooting of each image by the shooting device, the first centripetal acceleration and the second centripetal acceleration being centripetal accelerations of the shooting device in different coordinate systems, so as to extract centripetal accelerations at different angles, calculate a rotation axis of the shooting device in a world coordinate system according to the first centripetal acceleration, so as to determine a rotation axis corresponding to the target video through the coordinate system of the first centripetal acceleration, calculate a spatial rotation amount of the shooting device in the world coordinate system according to the rotation axis and the second centripetal acceleration, so as to calculate the coordinate system conversion relationship between the coordinate system of the second centripetal acceleration and the world coordinate system through the rotation axis and the second centripetal acceleration, perform spatial conversion on each image according to the spatial rotation amount, so as to obtain each clip perspective image and realize the spatial change of the image, and combine each clip perspective image, so as to obtain the target video. The method can automatically identify a target object surrounding the shooting, so as to efficiently generate a target video surrounding the target object, and the target video can realize a slow-motion special effect. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 An application environment diagram of the video generation method in one embodiment;

[0059] Figure 2 A flowchart of the video generation method in one embodiment;

[0060] Figure 3 A surrounding trajectory diagram when the rotation axis is parallel to the direction of gravity in one embodiment;

[0061] Figure 4 A surrounding trajectory diagram when the rotation axis is perpendicular to the direction of gravity in one embodiment;

[0062] Figure 5 This is a schematic diagram illustrating an application scenario of the video generation method in one embodiment;

[0063] Figure 6 This is a structural block diagram of a video generation device in one embodiment;

[0064] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] The video generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.

[0067] The terminal 102 can be, but is not limited to, various shooting devices. Shooting devices can be panoramic shooting devices, action shooting devices, or other shooting devices, such as cameras, drones, personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0068] In one embodiment, such as Figure 2 As shown, a method for generating target videos is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0069] Step 202: Obtain images obtained by the shooting device around the target object, and determine the first centripetal acceleration and the second centripetal acceleration of the shooting device when shooting each image; the first centripetal acceleration and the second centripetal acceleration are the centripetal accelerations of the shooting device in different coordinate systems.

[0070] The target object is a preset category of object that the photographing device orbits to photograph, and the object is not necessarily limited by the positional relationship between the photographing device and the selfie stick. Illustratively, the object can be a real person, the object can be a virtual person, and the object can also be an object or an animal.

[0071] The image obtained by the photographing device orbiting to photograph the target object is an image of the target object photographed by the photographing device during the orbiting process. A plurality of images of the target object are obtained by the photographing device orbiting to photograph the target object. During photographing of each image, the photographing device performs circular motion around the target object and photographs the target object. Illustratively, the photographing device can perform circular motion around the target object as the center on a horizontal plane and photograph the target object. The photographing device can also perform circular motion along a horizontal plane where the gravity direction of the target object is located and photograph the target object, and the center of the circular motion is the target object.

[0072] The first centripetal acceleration and the second centripetal acceleration are centripetal accelerations in different coordinate systems, and the first centripetal acceleration and the second centripetal acceleration are obtained through coordinate system conversion and corresponding processing. The first centripetal acceleration is a centripetal acceleration in a world coordinate system, and the world coordinate system is a global coordinate system. The second centripetal acceleration is a centripetal acceleration in a camera coordinate system, and the camera coordinate system is a local coordinate system. When the first centripetal acceleration is a centripetal acceleration in the world coordinate system, the first centripetal acceleration is a sine signal of acceleration. When the second centripetal acceleration is a centripetal acceleration in the camera coordinate system, the direction of the first centripetal acceleration is from the position of the photographing device to the direction of the rotation center of the photographing device orbiting to photograph.

[0073] In different coordinate systems, the components of each direction have their own frequencies. Illustratively, if the first centripetal acceleration is a centripetal acceleration in the world coordinate system, the first centripetal acceleration is a high-frequency component of acceleration, and the gravity is a low-frequency component of the acceleration during image photographing of the photographing device in the world coordinate system. If the second centripetal acceleration is a centripetal acceleration in the camera coordinate system, the second centripetal acceleration is a low-frequency component of acceleration, and the gravity is a high-frequency component of the acceleration during image photographing of the photographing device in the world coordinate system.

[0074] In one embodiment, the photographing device represents the photographing device. Correspondingly, determining the corresponding first centripetal acceleration and the second centripetal acceleration when the photographing device photographs each image includes: determining the acceleration of the photographing device in the camera coordinate system during the orbiting photographing; determining the acceleration of the photographing device in the world coordinate system according to the rotation matrix and the acceleration of the photographing device in the camera coordinate system; and extracting the components of the acceleration of the photographing device in the world coordinate system to obtain the corresponding first centripetal acceleration and the second centripetal acceleration when the photographing device photographs each image.

[0075] The acceleration of the photographing device in the camera coordinate system is measured by the gyroscope, and the rotation matrix is an anti-shake matrix based on an inertial measurement unit (IMU) for anti-shake processing, so that the acceleration is processed for anti-shake at the same time of the coordinate system conversion.

[0076] According to the rotation matrix and the acceleration of the photographing device in the camera coordinate system, the influence of the shaking of the acceleration of the photographing device in the world coordinate system caused by the environment and other factors is reduced, so as to accurately obtain the acceleration of the photographing device in the world coordinate system, and then the component extraction is performed on the acceleration of the photographing device in the world coordinate system, to obtain the first centripetal acceleration and the second centripetal acceleration corresponding to the photographing of each image by the photographing device. It can be understood that the acceleration of the photographing device in the world coordinate system is a signal with a frequency, and the first centripetal acceleration is a component signal of the signal.

[0077] In one embodiment, according to the rotation matrix and the acceleration of the photographing device in the camera coordinate system, the acceleration of the photographing device in the world coordinate system is determined, and the following formula is used:

[0078]

[0079] wherein a w is the acceleration of the photographing device in the world coordinate system, is the rotation matrix, and a is the measured acceleration.

[0080] In one embodiment, the method comprises the step of generating the rotation matrix: obtaining the time stamp and angular velocity corresponding to each image when the photographing device photographs each image; generating the rotation amount between the camera coordinate system and the world coordinate system according to the acceleration of the photographing device in the camera coordinate system and the angular velocity; and generating the rotation matrix based on the rotation amount between the camera coordinate system and the world coordinate system.

[0081] wherein the obtaining of the time stamp and angular velocity corresponding to each image when the photographing device photographs each image comprises: measuring the time stamp and camera angular velocity of the photographing device when photographing each image based on the gyroscope.

[0082] In one embodiment, the generation of the rotation matrix based on the rotation amount between the camera coordinate system and the world coordinate system comprises: performing smoothing processing on the rotation amount to obtain the rotation matrix when the time stamp of the photographing device is synchronized with the time stamp corresponding to each image. The smoothing processing can be a pose smoothing processing or other filtering method, and the pose smoothing processing includes but is not limited to a quaternion interpolation method.

[0083] In an optional embodiment, the acceleration of the photographing device in the camera coordinate system is an acceleration meter value; and the rotation amount between the camera coordinate system and the world coordinate system is generated according to the acceleration of the photographing device in the camera coordinate system and the angular velocity of the camera, comprising: estimating the rotation amount between the camera coordinate system and the world coordinate system by using an extended Kalman filter in combination with the acceleration meter value and the angular velocity value.

[0084] Exemplarily, the estimation of the rotation amount between the camera coordinate system and the world coordinate system by using the extended Kalman filter in combination with the acceleration meter value and the angular velocity value comprises: obtaining an initial state rotation amount and an initial process covariance; calculating a state transition matrix of the kth moment by using the angular velocity value; calculating a covariance matrix of state noise, updating a state rotation prior estimate and a process covariance prior estimate matrix; updating a noise variance matrix of an observation by using the acceleration meter value, calculating an observation transition Jacobian matrix, calculating an error between a current observation and an estimated observation; updating an optimal Kalman gain matrix of the kth moment; updating a camera coordinate system to world coordinate system rotation posterior estimate and a process covariance posterior estimate matrix according to the optimal Kalman gain matrix of the kth moment and the observation error; and calculating a rotation amount of the kth moment based on the camera coordinate system to world coordinate system rotation posterior estimate and the process covariance posterior estimate matrix.

[0085] The timestamps of the photographing device are synchronized with the timestamps of the images, comprising: synchronizing the gyroscope timestamps with the timestamps of the images, so that t k ≥ t j > t k-1 , wherein t j is the timestamp of each image, t k is the timestamp of the Kth frame of the gyroscope, and t k-1 is the timestamp of the K-1th frame of the gyroscope.

[0086] The state of the gyroscope is generated based on the relative rotation amount of adjacent timestamps, comprising: calculating the relative rotation amount of adjacent gyroscope timestamps based on a state posterior estimate of adjacent timestamps; performing quaternion interpolation to obtain the relative rotation amount of each image to the kth frame, and generating a rotation matrix of the jth image based on the relative rotation amount of each image to the kth frame.

[0087] The rotation amount of the photographing device to the world coordinate system is subjected to posture smoothing processing in the form of quaternion interpolation, etc., to generate a rotation matrix of the corresponding photographing device, so that a more accurate rotation matrix can be obtained. The panoramic image is rotated according to the current rotation matrix, so that each image is converted into a stable video frame. Therefore, the final video frame can be stabilized, and VR motion sickness can be reduced, which has strong robustness to large noise scenes and most motion scenes.

[0088] In one embodiment, the acceleration of the photographing device in the world coordinate system is determined according to the rotation matrix and the acceleration of the photographing device in the camera coordinate system, comprising: combining the rotation matrix and the acceleration of the photographing device in the camera coordinate system to realize coordinate system conversion of the acceleration, and obtaining the acceleration of the photographing device in the world coordinate system.

[0089] In one embodiment, the first centripetal acceleration and the second centripetal acceleration corresponding to the photographing of each image by the photographing device are obtained by component extraction of the acceleration of the photographing device in the world coordinate system, comprising: frequency selection of the components of the acceleration of the photographing device in the world coordinate system; determining whether the world coordinate system acceleration is preprocessed by filtering and anti-shake according to the range to which the selected frequency belongs; determining the calculation strategy of the first centripetal acceleration and the second centripetal acceleration according to whether the world coordinate system acceleration is preprocessed; and generating the first centripetal acceleration and the second centripetal acceleration according to the calculation strategy.

[0090] The frequency selection of the components of the acceleration of the photographing device in the world coordinate system comprises: selecting the frequency corresponding to one kind of centripetal acceleration in the first centripetal acceleration and the second centripetal acceleration; wherein the first centripetal acceleration is the centripetal acceleration in the world coordinate system, and the second centripetal acceleration is the centripetal acceleration in the camera coordinate system; and correspondingly, determining whether the world coordinate system acceleration is preprocessed by filtering and anti-shake according to the range to which the selected frequency belongs, comprising: when the first centripetal acceleration is selected, eliminating the influence of environmental factors on the world coordinate system acceleration to obtain the preprocessed world coordinate system acceleration; and when the second centripetal acceleration is selected, not pre-processing the world coordinate system acceleration.

[0091] In one embodiment, the first centripetal acceleration and the second centripetal acceleration are generated by a first calculation strategy with less calculation amount. The first centripetal acceleration and the second centripetal acceleration of the photographing device are obtained by component extraction of the acceleration of the photographing device in the world coordinate system, comprising: high-pass filtering the acceleration of the photographing device in the world coordinate system to obtain the first centripetal acceleration; and calculating the second centripetal acceleration according to the rotation matrix and the first centripetal acceleration.

[0092] In one embodiment, the first centripetal acceleration is obtained by high-pass filtering the acceleration of the photographing device in the world coordinate system, comprising: determining the cutoff frequency of the high-pass filter; and screening the acceleration exceeding the cutoff frequency of the high-pass filter to obtain the first centripetal acceleration. Illustratively, when the cutoff frequency is 0.05 Hz, the centripetal acceleration with a frequency exceeding 0.05 Hz is screened, and the centripetal acceleration with a frequency exceeding 0.05 Hz is determined as the first centripetal acceleration.

[0093] In one embodiment, the second centripetal acceleration is calculated according to the rotation matrix and the first centripetal acceleration, including: performing vector combination according to the rotation matrix and the first centripetal acceleration to obtain the second centripetal acceleration.

[0094] In one embodiment, the first centripetal acceleration and the second centripetal acceleration are generated by a first calculation strategy, and the formula used is as follows:

[0095]

[0096]

[0097]

[0098] wherein a w is the acceleration of the shooting device in the world coordinate system, is the rotation matrix, and a is the measured acceleration; represents the first centripetal acceleration; H(x) represents a high-pass filter; represents the second centripetal acceleration.

[0099] In one embodiment, a second calculation strategy capable of obtaining the first centripetal acceleration after noise reduction is used. The first centripetal acceleration and the second centripetal acceleration of the shooting device are obtained by performing component extraction on the acceleration of the shooting device in the world coordinate system, including: determining the first centripetal acceleration according to the acceleration of the shooting device in the world coordinate system and the gravitational acceleration; filtering the fusion result between the first centripetal acceleration and the rotation matrix by a low-pass filter to obtain the second centripetal acceleration; and performing noise reduction on the first centripetal acceleration according to the rotation matrix and the second centripetal acceleration to obtain the first centripetal acceleration after noise reduction. Correspondingly, the rotation axis of the shooting device in the world coordinate system is calculated according to the first centripetal acceleration, including: calculating the rotation axis of the shooting device in the world coordinate system according to the first centripetal acceleration after noise reduction.

[0100] In one embodiment, the first centripetal acceleration is determined according to the acceleration of the shooting device in the world coordinate system and the gravitational acceleration, including: determining the vector form of the acceleration of the shooting device in the world coordinate system and the gravitational acceleration in the form of a multi-dimensional vector; and combining the acceleration of the shooting device in the world coordinate system and the gravitational acceleration according to the vector form to obtain the first centripetal acceleration to be reduced.

[0101] In one embodiment, the fusion result between the first centripetal acceleration and the rotation matrix is filtered by a low-pass filter, including: generating the fusion result of the first centripetal acceleration and the rotation matrix; determining a cutoff frequency of the low-pass filter; screening the fusion result lower than the cutoff frequency of the low-pass filter to obtain a second centripetal acceleration. Exemplarily, the cutoff frequency is 0.05 Hz, and the fusion result lower than 0.05 Hz is screened to obtain the second centripetal acceleration.

[0102] In one embodiment, the first centripetal acceleration is denoised according to the rotation matrix and the second centripetal acceleration to obtain a denoised first centripetal acceleration, including: transposing the rotation matrix, combining the transposed rotation matrix and the second centripetal acceleration to obtain the denoised first centripetal acceleration, and the accuracy of the denoised first centripetal acceleration is higher than that of the first centripetal acceleration without denoising.

[0103] In one embodiment, the first centripetal acceleration and the second centripetal acceleration are generated by a second calculation strategy, and the formula adopted is as follows:

[0104]

[0105]

[0106]

[0107]

[0108] wherein a w is the acceleration of the shooting device in the world coordinate system, is the rotation matrix, and a is the measured acceleration; represents the first centripetal acceleration; g w is the gravity vector; L(X) represents a low-pass filter; represents the second centripetal acceleration, represents the denoised first centripetal acceleration.

[0109] Step 204, the rotation axis of the shooting device in the world coordinate system is calculated according to the first centripetal acceleration.

[0110] The world coordinate system is the coordinate system in which the target video is located; the coordinate system in which the first centripetal acceleration is located can be the world coordinate system. Exemplarily, the world coordinate system is used to select a reference coordinate system in the environment to describe the positions of the camera and the object.

[0111] The rotation axis is an axis in the world coordinate system during the process of the photographing device revolving around the photographing target object, and the axis can be represented by a vector in the world coordinate system to calculate the rotation axis of the photographing device in the world coordinate system according to the first centripetal acceleration. Exemplarily, the direction of the world coordinate system rotation axis is calculated by using the following formula:

[0112]

[0113] wherein, represents the first centripetal acceleration; k w represents the target rotation axis meeting the minimum value condition, and k represents the candidate rotation axis of the photographing device in the world coordinate system.

[0114] In one embodiment, the rotation axis of the photographing device in the world coordinate system is calculated according to the first centripetal acceleration, including: generating a combination value of the candidate rotation axis according to the candidate rotation axis of the world coordinate system and the first centripetal acceleration at each time; and selecting the target rotation axis of the world coordinate system from the candidate rotation axis according to the combination value. Correspondingly, the spatial rotation amount of the photographing device in the world coordinate system is calculated according to the rotation axis and the second centripetal acceleration, including: calculating the spatial rotation amount of the photographing device in the world coordinate system according to the target rotation axis and the second centripetal acceleration.

[0115] In one embodiment, the combination value of the candidate rotation axis is generated according to the candidate rotation axis in the world coordinate system and the first centripetal acceleration of each image, including: respectively determining each candidate rotation axis in the world coordinate system; calculating the combination value of the first centripetal acceleration of each image and the corresponding candidate rotation axis under each candidate rotation axis; wherein each candidate rotation axis corresponds to the first centripetal acceleration at each time; and selecting the target rotation axis from each candidate rotation axis according to the combination value.

[0116] In one embodiment, the target rotation axis is selected from the candidate rotation axis according to the combination value, including: comparing the combination values of each candidate rotation axis to determine that the candidate rotation axis with the minimum combination value is the target rotation axis.

[0117] Step 206, the spatial rotation amount of the photographing device in the world coordinate system is calculated according to the rotation axis and the second centripetal acceleration.

[0118] The spatial rotation amount is used to convert each image to a corresponding spatial dimension to generate at least part of a panoramic video or other types of target videos. The spatial rotation amount of the photographing device in the world coordinate system includes multiple vectors with different directions, and these vectors are combined with each image to convert the planar image to at least part of the perspective image in the target video.

[0119] In one embodiment, the image taken by the photographing device around the target object is processed, and the generation mode of the spatial rotation quantity is adjusted along with the plane where the circumferential motion is located, so as to adapt to the image generation target video of the images taken at different angles. The expression of the spatial rotation quantity is as follows:

[0120]

[0121] wherein, is a spatial rotation quantity, which can be represented by a spatial rotation matrix; is a first column vector of the spatial rotation quantity, and is a first vector; is a second column vector of the spatial rotation quantity, is a third column vector of the spatial rotation quantity; the second column vector and the third column vector respectively act as a second vector and a third vector according to whether the rotation axis is parallel to the direction of gravity.

[0122] The spatial rotation quantity includes the first vector, the second vector and the third vector with different directions; the spatial rotation quantity is calculated according to the rotation axis and the second centripetal acceleration, including: generating the first vector according to the second centripetal acceleration; judging whether to generate the second vector based on the rotation matrix according to the included angle between the target rotation axis and the direction of gravity; if yes, generating the second vector based on the first vector, the target rotation axis and the rotation matrix, combining the first vector and the second vector to obtain the third vector; if no, generating the second vector based on the first vector and a preset matrix, combining the first vector and the second vector to obtain the third vector.

[0123] In one embodiment, the first vector is generated according to the second centripetal acceleration, including: determining the vector with the size of the basis vector as the first vector according to the direction of the second centripetal acceleration.

[0124] In one embodiment, whether to generate the second vector based on the rotation matrix is judged according to the included angle between the target rotation axis and the direction of gravity, including: determining the vector fusion result corresponding to the target rotation axis and the direction of gravity, calculating the size of the vector fusion result and a preset threshold; judging whether to generate the second vector based on the rotation matrix based on whether the vector fusion result exceeds the preset threshold. Exemplarily, the vector fusion result can be the norm of the vector fusion, when the norm is less than the preset threshold, the target rotation axis and the direction of gravity are parallel, and there is no included angle between them, so the second vector does not need to be generated based on the rotation matrix; when the norm is greater than the preset threshold, the target rotation axis and the direction of gravity are perpendicular, and there is no included angle between them, so the second vector is generated based on the rotation matrix.

[0125] In one embodiment, the second vector generated based on the rotation matrix is not necessarily the same as the vector of the world coordinate system. When there is no included angle between the target rotation axis and the gravity direction or the included angle is less than a corresponding threshold value, the rotation axis is parallel to the gravity direction, the second vector is the second column vector of the spatial rotation quantity, and the third vector is the third column vector of the spatial rotation quantity. Correspondingly, when the target rotation axis is parallel to the gravity direction, the second vector is generated based on the first vector and a preset matrix, and the third vector is obtained by combining the first vector and the second vector, including: fusing the first vector and the preset matrix to obtain the second vector; and fusing the first vector and the second vector to obtain the third vector. Exemplarily, the first vector is an x-axis vector in the world coordinate system, the second vector is a y-axis vector in the world coordinate system, and the third vector is a z-axis vector in the world coordinate system, as shown in Figure 3

[0126] In Figure 3 , the condition that the rotation axis is parallel to the gravity direction is as follows:

[0127] ||kw×gw||<σ;

[0128] wherein, k w is the target rotation axis; g w is the gravity direction; and σ is a preset threshold value.

[0129] Figure 3 The corresponding formula is as follows:

[0130]

[0131]

[0132]

[0133] wherein, is the first vector, is the second vector, is the third vector, and is a preset vector.

[0134] ​In one embodiment, when the included angle between the target rotation axis and the gravity direction exceeds a corresponding threshold value, the rotation axis is perpendicular to the gravity direction, the second vector is a third column vector of the spatial rotation quantity, and the third vector is a second column vector of the spatial rotation quantity. Correspondingly, the second vector is generated based on the rotation matrix; the third vector is obtained by combining the first vector and the second vector, including: fusing the target rotation axis and the rotation matrix to obtain a fusion result between the target rotation axis and the rotation matrix; combining the fusion result and the first vector to obtain the second vector; and combining the first vector and the second vector to obtain the third vector. Exemplarily, the first vector is an x-axis vector in the world coordinate system, the second vector is a z-axis vector in the world coordinate system, and the third vector is a y-axis vector in the world coordinate system, as shown in FIG. 3. Figure 4

[0135] In the above embodiment, the condition that the rotation axis is perpendicular to the gravity direction is as follows: Figure 4

[0136] ||kw×gw||>σ;

[0137] wherein, k w is the target rotation axis; g w is the gravity direction; and σ is a preset threshold value.

[0138] Figure 4 The corresponding formula is as follows:

[0139]

[0140]

[0141]

[0142] wherein, is the first vector, is the second vector, is the third vector, is the rotation matrix, and k w is the target rotation axis.

[0143] In step 208, each image is spatially converted according to each spatial rotation quantity to obtain each clip perspective image.

[0144] ​​In one embodiment, the space conversion of each image according to the spatial rotation amount comprises: determining a mapping relationship between each photographed image and a panoramic view angle planar image, mapping each photographed image based on the mapping relationship to obtain a panoramic view angle planar image corresponding to each photographed image; and performing color interpolation resampling on each panoramic view angle planar image to obtain each clip view angle image. The specific implementation of the color interpolation resampling can be a quadratic linear sampling or the like.

[0145] The generation process of the mapping relationship comprises: projecting an output image grid point coordinate into a camera coordinate system 3D coordinate; rotating the 3D coordinate to a world coordinate system; converting a 3D point of the world coordinate system into a 2D planar coordinate of the target video after the anti-shake; and calculating the mapping relationship of all output image planar grid points according to the above steps to generate a mapping map.

[0146] Exemplarily, the process of the space conversion of each image according to the spatial rotation amount comprises: projecting an output image grid point coordinate into a camera coordinate system 3D coordinate; rotating the 3D coordinate to a world coordinate system according to the positional relationship between the shooting device and the IMU; converting a 3D point of the world coordinate system into a 2D planar coordinate of the target video after the anti-shake; calculating the mapping relationship of all output image planar grid points according to the above steps to generate a mapping map; performing color interpolation resampling on the target video image using the mapping map to obtain a panoramic view angle planar image corresponding to each photographed image; and performing quadratic linear sampling processing on the color of each panoramic view angle planar image to obtain each clip view angle image.

[0147] Step 210: combining each clip view angle image to obtain the target video.

[0148] In one embodiment, the combination of each clip view angle image comprises: calculating a first angle based on the first vector and the second vector of the first clip view angle image; the first angle is used to represent the clip time corresponding to the first clip view angle image; calculating a second angle based on the first vector and the second vector of the second clip view angle image; the second angle is used to represent the clip time corresponding to the second clip view angle image; calculating an angle difference between the first angle and the second angle; and combining the first clip view angle image and the second clip view angle image according to the angle difference.

[0149] The first clip view angle image and the second clip view angle image are generated by images photographed by the shooting device at different positions of the same circular motion. The first angle can be calculated by the first vector and the second vector of the first clip view angle image.

[0150] The first vector of the first and second clip perspective images is a base vector generated in the respective second acceleration direction, and the second vector of the first and second clip perspective images varies with the target rotation axis and the gravity direction; when the target rotation axis is parallel to the gravity direction, the second vector is the second column vector of the spatial rotation quantity; and when the target rotation axis is perpendicular to the gravity direction, the second vector is the third column vector of the spatial rotation quantity. The first and second angles are generated by performing inverse trigonometric function operations on the respective first and second vectors.

[0151] Specifically, when the target rotation axis is parallel to the gravity direction, the calculation formulas of the first and second angles are the same, and are specifically as follows:

[0152]

[0153] wherein, is the second element of the spatial rotation matrix column vector; is the first element of the spatial vector;

[0154] Specifically, when the target rotation axis is perpendicular to the gravity direction, the calculation formulas of the first and second angles are as follows:

[0155]

[0156] wherein, is the third element of the spatial rotation matrix column vector; and is the projection of the spatial rotation matrix column vector on the target object vector around which the shooting device is shooting, wherein:

[0157]

[0158]

[0159] is a vector perpendicular to both the gravity direction (or upward direction) and the target rotation axis direction.

[0160] In one embodiment, the combination of the first and second clip perspective images according to the angle difference value comprises: comparing the angle difference value with a preset angle value representing the number of circumferential movements, to determine whether the angle difference value corresponds to a perspective point of completed rotation; if so, it is determined that the first and second clip perspective images are associated frames of the target video, and the associated frames are spliced to seamlessly connect the clip perspective images to form the target video. It can be understood that the clip perspective images are spliced in the time dimension during the combination of the clip perspective images.

[0161] Correspondingly, it is judged whether the angle difference corresponds to a rotation completed view angle point, and the formula is one or more of the following formulas:

[0162] ||fmod(θ s -θ e , 2π)||<σ;

[0163] ||fmod(θ S -θ e , 2π)||>2π-σ;

[0164] Wherein, θ S is the first angle, θ E， is the second angle, ||fmod(θ S -θ e , 2π)|| represents the norm of the difference of θ s -θ e and the difference value of 2π, and 2π represents the angle of completing a circumferential motion.

[0165] In the above video generation method, the images obtained by the shooting device surrounding the shooting target object are acquired, and the corresponding first centripetal acceleration and second centripetal acceleration of the shooting device when shooting each image are determined; the first centripetal acceleration and the second centripetal acceleration are the centripetal accelerations of the shooting device under different coordinate systems, so as to extract the centripetal acceleration at different angles; the rotation axis of the shooting device under the world coordinate system is calculated according to the first centripetal acceleration, so as to determine the rotation axis corresponding to the target video through the coordinate system of the first centripetal acceleration; the spatial rotation amount is calculated according to the rotation axis and the second centripetal acceleration, so as to calculate the coordinate system conversion relationship between the second centripetal acceleration coordinate system and the world coordinate system through the rotation axis and the second centripetal acceleration; each image is spatially converted according to the spatial rotation amount, to obtain each clip perspective image, so as to realize the spatial change of the image; and each clip perspective image is combined to obtain the target video. By using the method, the target object of the surrounding shooting can be automatically recognized, so that a target video of the target object shot in a surrounding manner can be efficiently generated, and the target video can realize a slow-motion special effect.

[0166] In one embodiment, as Figure 5As shown, the panoramic video is taken as the target video, and the generation process of the target video is discussed from the whole process, which includes: acquiring images and IMU data measured by a gyroscope, generating a rotation matrix according to the IMU data, performing anti-shake processing on the acquired images according to the rotation matrix to obtain anti-shake images; then generating a first centripetal acceleration and a second centripetal acceleration according to related data in the IMU data, generating a space rotation quantity based on the first centripetal acceleration and the second centripetal acceleration, and the space rotation quantity can also be called a view rotation matrix; generating a clip view image based on the space rotation quantity, time-aligning the clip view images so that the positions of the first clip view images are the same, and forming the panoramic video.

[0167] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0168] Based on the same inventive concept, the embodiments of the present application also provide a video generation device for implementing the above-mentioned video generation method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more video generation device embodiments provided below can refer to the limitations of the video generation method in the above text, which will not be repeated here.

[0169] In one embodiment, as shown in Figure 6 A video generation device is provided, comprising: a data acquisition module 602, a rotation axis calculation module 604, a space rotation calculation module 606, an image conversion module 608, and a video generation module 610, wherein:

[0170] The data acquisition module 602 is configured to acquire images obtained by a shooting device surrounding a shooting target object, and determine a first centripetal acceleration and a second centripetal acceleration corresponding to the shooting of each of the images by the shooting device; the first centripetal acceleration and the second centripetal acceleration are centripetal accelerations of the shooting device in different coordinate systems;

[0171] The rotation axis calculation module 604 is configured to calculate a rotation axis of the shooting device in a world coordinate system according to the first centripetal acceleration;

[0172] The spatial rotation calculation module 606 is configured to calculate a spatial rotation of the shooting device in a world coordinate system according to the rotation axis and the second centripetal acceleration.

[0173] The image conversion module 608 is configured to perform spatial conversion on each of the images according to each of the spatial rotations to obtain each of the clip perspective images.

[0174] The target video generation module 610 is configured to combine each of the clip perspective images to obtain the target video.

[0175] In one of the embodiments, the data acquisition module 602 is configured to:

[0176] determine an acceleration of the shooting device in a camera coordinate system when the surround shooting is performed;

[0177] determine an acceleration of the shooting device in a world coordinate system according to the rotation matrix and the acceleration of the shooting device in the camera coordinate system;

[0178] perform component extraction on the acceleration of the shooting device in the world coordinate system to obtain a first centripetal acceleration and a second centripetal acceleration of the shooting device.

[0179] In one of the embodiments, the data acquisition module 602 includes:

[0180] determine a first centripetal acceleration according to the acceleration of the shooting device in the world coordinate system and a gravitational acceleration;

[0181] perform filtering processing on a fusion result between the first centripetal acceleration and the rotation matrix through a low-pass filter to obtain a second centripetal acceleration;

[0182] perform noise reduction on the first centripetal acceleration according to the rotation matrix and the second centripetal acceleration to obtain a noise-reduced first centripetal acceleration;

[0183] Correspondingly, the rotation axis calculation module 604 is configured to calculate a rotation axis of the shooting device in the world coordinate system according to the noise-reduced first centripetal acceleration.

[0184] In one of the embodiments, the data acquisition module 602 is specifically configured to:

[0185] perform high-pass filtering on the acceleration of the shooting device in the world coordinate system to obtain a first centripetal acceleration;

[0186] determine a second centripetal acceleration according to the rotation matrix and the first centripetal acceleration.

[0187] In one of the embodiments, the data acquisition module 602 is further configured to:

[0188] acquire a timestamp and an angular velocity corresponding to each of the images when the photographing device photographs each of the images;

[0189] generate a rotation amount between a camera coordinate system and a world coordinate system according to the acceleration of the photographing device in the camera coordinate system and the angular velocity;

[0190] generate the rotation matrix based on the rotation amount between the camera coordinate system and the world coordinate system.

[0191] In one of the embodiments, the data acquisition module 602 is configured to perform smoothing processing on the rotation amount to obtain the rotation matrix when the timestamp of the camera is synchronized with the timestamp corresponding to each of the images.

[0192] In one of the embodiments, the rotation axis calculation module 604 is specifically configured to:

[0193] calculate a combination value of the candidate rotation axis according to the candidate rotation axis of the photographing device in the world coordinate system and the first centripetal acceleration corresponding to each of the images when the photographing device photographs each of the images;

[0194] determine the target rotation axis of the photographing device in the world coordinate system from the candidate rotation axis according to the combination value;

[0195] The spatial rotation calculation module 606 is configured to calculate the spatial rotation amount of the photographing device in the world coordinate system according to the target rotation axis and the second centripetal acceleration.

[0196] In one of the embodiments, the spatial rotation amount includes first, second and third vectors with different directions; and the spatial rotation calculation module 606 is configured to:

[0197] generate the first vector according to the second centripetal acceleration;

[0198] determine whether to generate the second vector based on the rotation matrix according to an included angle between the target rotation axis and the direction of gravity;

[0199] if yes, generate the second vector based on the first vector, the target rotation axis and the rotation matrix, combine the first vector and the second vector to obtain the third vector;

[0200] if no, generate the second vector based on the first vector and a preset matrix, and combine the first vector and the second vector to obtain the third vector.

[0201] In one of the embodiments, the image conversion module 608 is configured to determine a mapping relationship between each photographed image and a panoramic perspective planar image.

[0202] map each image based on the mapping relationship to obtain a panoramic perspective planar image corresponding to each image;

[0203] perform color interpolation resampling on each panoramic perspective planar image to obtain each clip perspective image.

[0204] In one of the embodiments, the clip perspective image includes a first clip perspective image and a second clip perspective image; the video generation module 610 is configured to:

[0205] calculate a first angle based on a first vector and a second vector of the first clip perspective image; the first angle is used to represent a clip time corresponding to the first clip perspective image;

[0206] calculate a second angle based on a first vector and a second vector of the second clip perspective image; the second angle is used to represent a clip time corresponding to the second clip perspective image;

[0207] calculate an angle difference between the first angle and the second angle;

[0208] combine the first clip perspective image and the second clip perspective image according to the angle difference.

[0209] Each module in the above video generation apparatus can be realized by software, hardware, and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0210] In one embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 7The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a video generation method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. It can also be an external keyboard, touchpad or mouse, etc.

[0211] Those skilled in the art can understand that, Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0212] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the steps in each of the above method embodiments.

[0213] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to realize the steps in each of the above method embodiments.

[0214] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to realize the steps in each of the above method embodiments.

[0215] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0216] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0217] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is considered to be within the scope of the present disclosure. The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of video generation, the method comprising: The method comprises: obtaining images obtained by a shooting device surrounding a shooting target object, and determining first centripetal acceleration and second centripetal acceleration corresponding to the shooting device shooting each image; the first centripetal acceleration and the second centripetal acceleration are centripetal accelerations of the shooting device in different coordinate systems; calculating a rotation axis of the shooting device in a world coordinate system according to the first centripetal acceleration; calculating a spatial rotation amount of the shooting device in the world coordinate system according to the rotation axis and the second centripetal acceleration; performing spatial conversion on each image according to each spatial rotation amount to obtain each cut visual angle image; combining each cut visual angle image to obtain a target video.

2. The method of claim 1, wherein, The method comprises: determining acceleration of the shooting device in a camera coordinate system when surrounding shooting is performed; determining acceleration of the shooting device in a world coordinate system according to a rotation matrix and the acceleration of the shooting device in the camera coordinate system; extracting components and converting coordinate systems of the acceleration of the shooting device in the world coordinate system to obtain first centripetal acceleration and second centripetal acceleration of the shooting device.

3. The method of claim 2, wherein, The method comprises: determining first centripetal acceleration according to the acceleration of the shooting device in the world coordinate system and gravitational acceleration; filtering a fusion result between the first centripetal acceleration and the rotation matrix through a low-pass filter to obtain second centripetal acceleration; determining first centripetal acceleration according to the rotation matrix and the second centripetal acceleration to obtain first centripetal acceleration after noise reduction; The method comprises: calculating the rotation axis of the shooting device in the world coordinate system according to the first centripetal acceleration after noise reduction.

4. The method of claim 2, wherein, The method comprises: performing high-pass filtering on the acceleration of the shooting device in the world coordinate system to obtain first centripetal acceleration; determining second centripetal acceleration according to the rotation matrix and the first centripetal acceleration.

5. The method according to any one of claims 2 to 4, characterized in that, The method further comprises: obtaining time stamps and angular velocities corresponding to each image when the shooting device shoots each image; generating a rotation amount between a camera coordinate system and a world coordinate system according to the acceleration of the shooting device in the camera coordinate system and the angular velocities; generating the rotation matrix based on the rotation amount between the camera coordinate system and the world coordinate system.

6. The method of claim 5, wherein, The method comprises: performing smoothing processing on the rotation amount to obtain the rotation matrix when the time stamp of the shooting device is synchronized with the time stamps corresponding to each image.

7. The method of claim 1, wherein, The calculating the rotation axis of the photographing device in the world coordinate system according to the first centripetal acceleration comprises: According to the candidate rotation axis of the photographing device in the world coordinate system and the first centripetal acceleration corresponding to the photographing of each image, a combination value of the candidate rotation axis is calculated; According to the combination value, a target rotation axis of the photographing device in the world coordinate system is determined from the candidate rotation axes; The calculating the spatial rotation amount of the photographing device in the world coordinate system according to the rotation axis and the second centripetal acceleration comprises: The calculating the spatial rotation amount of the photographing device in the world coordinate system according to the target rotation axis and the second centripetal acceleration.

8. The method of claim 1, wherein, The spatial rotation amount comprises first, second and third vectors with different directions; the calculating the spatial rotation amount of the photographing device in the world coordinate system according to the rotation axis and the second centripetal acceleration comprises: The first vector is generated according to the second centripetal acceleration; Whether the second vector is generated based on a rotation matrix is determined according to an included angle between the target rotation axis and the direction of gravity; If yes, the second vector is generated based on the first vector, the target rotation axis and the rotation matrix, and the first vector and the second vector are combined to obtain the third vector; If no, the second vector is generated based on the first vector and a preset matrix, and the first vector and the second vector are combined to obtain the third vector.

9. The method of claim 1, wherein, The spatial rotation amount comprises first and second vectors with different directions; the clipping perspective images comprise first and second clipping perspective images; and the combining the clipping perspective images comprises: A first angle is calculated based on the first and second vectors of the first clipping perspective image; the first angle is used to represent a clipping time corresponding to the first clipping perspective image; A second angle is calculated based on the first and second vectors of the second clipping perspective image; the second angle is used to represent a clipping time corresponding to the second clipping perspective image; An angle difference value between the first angle and the second angle is calculated; According to the angle difference value, the first clipping perspective image and the second clipping perspective image are combined.

10. The method of claim 1, wherein, The spatial conversion of each image according to the spatial rotation amount to obtain the clipping perspective images comprises: A mapping relationship between each photographed image and a panoramic perspective planar image is determined; Each image is mapped based on the mapping relationship to obtain a panoramic perspective planar image corresponding to each image; Color interpolation resampling is performed on each panoramic perspective planar image to obtain each clipping perspective image.

11. A video generating apparatus characterized by comprising: The device comprises: A data acquisition module is configured to acquire images obtained by a photographing device surrounding a photographing target object, and determine first and second centripetal accelerations corresponding to the photographing of each image by the photographing device; the first and second centripetal accelerations are centripetal accelerations of the photographing device in different coordinate systems. a rotation axis calculation module, configured to calculate a rotation axis of the photographing device in a world coordinate system according to the first centripetal acceleration; a space rotation calculation module, configured to calculate a space rotation amount of the photographing device in the world coordinate system according to the rotation axis and the second centripetal acceleration; an image conversion module, configured to perform space conversion on each of the images according to each of the space rotation amounts to obtain each of the clip perspective images; a target video generation module, configured to combine each of the clip perspective images to obtain a target video.

12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 10.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 10.

Images