Video uniform speed display method and device, computer readable medium and electronic device

Abstract

The present disclosure provides a method and device for uniform speed display of video, a computer-readable medium and an electronic device, and relates to the technical field of video anti-shake. The method comprises: obtaining a current image frame and current camera posture data corresponding to the current image frame; forming a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to multiple image frames after the current image frame; determining the smoothed camera posture data corresponding to the current image frame through the camera posture data sequence; performing uniform speed processing on the current image frame based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake. The present disclosure can solve the problem of fast and slow motion speed during camera shooting, and can make the motion speed of the captured video uniform, thereby improving the video quality.

Description

Background Art

[0002] With the rapid development of science and technology, people are increasingly fond of recording their lives through the cameras on their smartphones. Panoramic shooting is a common shooting method. Since it is handheld shooting, the photographer's hands will inevitably have random shaking and the rotation speed will be fast and slow. The video shot in this way will be randomly shaking and the movement speed will be fast and slow. The ideal panoramic video in the photographer's mind should be stable, without random shaking, and with uniform movement speed.

[0003] At present, in the relevant video processing solutions, either the uniform speed display of the video cannot be achieved, or when the video is processed at a uniform speed, analysis based on the image content is required, which has a large computational overhead, and the result of the uniform speed processing is affected by the quality of the image content, and the uniform speed processing result is unstable. Summary of the invention

[0004] The purpose of the present disclosure is to provide a video uniform speed display method, a video uniform speed display device, a computer-readable medium and an electronic device, so as to achieve uniform video motion speed at least to a certain extent, while reducing computing overhead and improving the stability of processing results.

[0005] According to a first aspect of the present disclosure, a method for uniformly displaying a video is provided, comprising:

[0006] Obtaining a current image frame and current camera posture data corresponding to the current image frame;

[0007] Constructing a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to a plurality of image frames after the current image frame;

[0008] Determine the smoothed camera pose data corresponding to the current image frame through the camera pose data sequence;

[0009] The current image frame is processed at a uniform speed based on the current camera posture data and the smoothed camera posture data to obtain a current image frame after uniform speed anti-shake.

[0010] According to a second aspect of the present disclosure, a video uniform speed display device is provided, comprising:

[0011] A current camera posture acquisition module is used to acquire a current image frame and current camera posture data corresponding to the current image frame;

[0012] A camera posture sequence construction module, used to construct a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to a plurality of image frames after the current image frame;

[0013] A smooth camera posture determination module, used to determine the smooth camera posture data corresponding to the current image frame through the camera posture data sequence;

[0014] The image frame uniform speed module is used to perform uniform speed processing on the current image frame based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake.

[0015] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0016] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0017] Processor; and

[0018] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0019] A video uniform speed display method provided by an embodiment of the present disclosure obtains the current image frame and the current camera posture data corresponding to the current image frame, and can form a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to multiple image frames after the current image frame, and then determine the smooth camera posture data corresponding to the current image frame through the camera posture data sequence, and finally perform uniform speed processing on the current image frame based on the current camera posture data and the smooth camera posture data to obtain the current image frame after uniform speed anti-shake. On the one hand, according to the camera posture data of multiple frames, the smooth camera posture data of the current image frame is calculated, and the uniform speed processing of the video is realized by the smooth camera posture data, and the processing result is not affected by the imaging quality of the camera, effectively reducing the error and making the processing result more stable; on the other hand, according to the current camera posture data of the current image frame and the camera posture data corresponding to multiple image frames after the current image frame, the smooth camera posture data corresponding to the current image frame is determined, the accuracy of the smooth camera posture data is improved, and the smoothness and coherence of the uniform speed result in the whole processing process are guaranteed, the video quality after uniform speed processing is improved, and the user experience is improved.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0023] Figure 2 A schematic diagram showing an electronic device to which the embodiments of the present disclosure can be applied;

[0024] Figure 3 A schematic diagram schematically illustrates a camera motion coordinate axis in an exemplary embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram schematically illustrates a flow chart of a method for uniformly displaying a video in an exemplary embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of a process for determining smoothed camera posture data in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 6 A schematic diagram of a process of cutting a current image frame in an exemplary embodiment of the present disclosure is schematically shown;

[0028] Figure 7 A schematic diagram of a process for updating smoothed camera posture data in an exemplary embodiment of the present disclosure is schematically shown;

[0029] Figure 8 Another schematic diagram of a process for updating smoothed camera posture data in an exemplary embodiment of the present disclosure is schematically shown;

[0030] Fig. 9 A schematic diagram of a process of performing uniform speed anti-shake on a current image frame in an exemplary embodiment of the present disclosure is schematically shown;

[0031] Fig.10 A schematic diagram schematically shows a principle of implementing uniform speed anti-shake in an exemplary embodiment of the present disclosure;

[0032] Fig.11 A schematic diagram of a process of displaying a video at a uniform speed in an exemplary embodiment of the present disclosure is schematically shown;

[0033] Fig.12 The following is a schematic diagram showing the composition of a video uniform speed display device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which a method and apparatus for uniformly displaying videos according to an embodiment of the present disclosure can be applied is shown.

[0037] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0038] The video uniform speed display method provided in the embodiment of the present disclosure is generally executed by the terminal devices 101, 102, and 103, and accordingly, the video uniform speed display device is generally set in the terminal devices 101, 102, and 103. However, it is easy for those skilled in the art to understand that the video uniform speed display method provided in the embodiment of the present disclosure can also be executed by the server 105, and accordingly, the video uniform speed display device can also be set in the server 105, which is not particularly limited in this exemplary embodiment.

[0039] For example, in an exemplary embodiment, a user may upload the current image frame captured, the current camera posture data collected, and the camera posture data corresponding to multiple image frames after the current image frame to the server 105 through the terminal devices 101, 102, and 103. The server generates the current image frame after uniform speed processing through the video uniform speed display method provided in the embodiment of the present disclosure, and transmits the current image frame after uniform speed processing to the terminal devices 101, 102, 103, etc. for display.

[0040] An exemplary embodiment of the present disclosure provides an electronic device for implementing a method for displaying a video at a uniform speed, which may be Figure 1 The terminal device 101, 102, 103 or the server 105 in the embodiment of the present invention comprises at least a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the video uniform speed display method by executing the executable instructions.

[0041] Below Figure 2 Taking the mobile terminal 200 in FIG. 1 as an example, the structure of the electronic device is exemplarily described. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Figure 2 The structure in the figure can also be applied to fixed type devices. In other embodiments, the mobile terminal 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also adopt the same Figure 2 Different interface connection methods, or a combination of multiple interface connection methods.

[0042] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0043] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0044] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the mobile terminal 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0045] The processor 210 is provided with a memory. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the execution is controlled by the processor 210.

[0046] The charging management module 240 is used to receive charging input from the charger. The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, the internal memory 221, the display screen 290, the camera module 291 and the wireless communication module 260.

[0047] The wireless communication function of the mobile terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modulation and demodulation processor and baseband processor. Among them, antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals; mobile communication module 250 can provide solutions for wireless communication including 2G / 3G / 4G / 5G applied on mobile terminal 200; modulation and demodulation processor can include modulator and demodulator; wireless communication module 260 can provide solutions for wireless communication including wireless local area network (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT) and the like applied on mobile terminal 200. In some embodiments, antenna 1 of mobile terminal 200 is coupled with mobile communication module 250, and antenna 2 is coupled with wireless communication module 260, so that mobile terminal 200 can communicate with network and other devices through wireless communication technology.

[0048] The mobile terminal 200 implements the display function through a GPU, a display screen 290, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0049] The mobile terminal 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290 and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera module 291; the camera module 291 is used to capture static images or videos; the digital signal processor is used to process digital signals, and can process other digital signals in addition to digital image signals; the video codec is used to compress or decompress digital videos, and the mobile terminal 200 can also support one or more video codecs.

[0050] The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external memory card communicates with the processor 210 through the external memory interface 222 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0051] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile terminal 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (Universal Flash Storage, UFS), etc. The processor 210 executes various functional applications and data processing of the mobile terminal 200 by running instructions stored in the internal memory 221 and / or instructions stored in a memory provided in the processor.

[0052] The mobile terminal 200 can implement audio functions such as music playing and recording through the audio module 270 , the speaker 271 , the receiver 272 , the microphone 273 , the earphone interface 274 and the application processor.

[0053] The depth sensor 2801 is used to obtain the depth information of the scene. In some embodiments, the depth sensor can be arranged in the camera module 291.

[0054] The pressure sensor 2802 is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 2802 can be arranged on the display screen 290. There are many types of pressure sensors 2802, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0055] The gyro sensor 2803 can be used to determine the motion posture of the mobile terminal 200. In some embodiments, the angular velocity of the mobile terminal 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 2803. The gyro sensor 2803 can be used for shooting anti-shake, navigation, somatosensory game scenes, etc.

[0056] In addition, sensors with other functions may be set in the sensor module 280 according to actual needs, such as an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0057] The mobile terminal 200 may also include other devices that provide auxiliary functions. For example, the key 294 includes a power key, a volume key, etc., and the user can generate key signal input related to the user settings and function control of the mobile terminal 200 through key input. For example, the indicator 292, the motor 293, the SIM card interface 295, etc.

[0058] At present, most terminal devices with image acquisition functions, such as smartphones, are equipped with inertial measurement units (IMUs). Generally speaking, IMUs are installed at the center of gravity of the object being measured. When running, IMUs can measure the angular velocity of the camera along the three axes in the camera coordinate system at the current moment.

[0059] For the convenience of explanation and understanding, the camera coordinate system in the embodiment of the present disclosure can be as follows: Figure 3 As shown, when the terminal device is in the default screen orientation, the X axis extends horizontally to the right, the Y axis extends vertically upward, and the Z axis extends perpendicular to the screen. It is easy to understand that in Figure 3 In the camera coordinate system shown, coordinates behind the screen will have negative Z values.

[0060] Among the related video stabilization technologies, one technical solution is to use an inertial sensor to track the rotation of the camera, estimate the camera translation through the coordinate changes of feature points in the video frame, and then fuse the rotational motion and translational motion of the camera to compensate for the camera motion to achieve video stabilization; another technical solution is to divide the video frame into many grids, estimate the motion of each grid point through feature point matching, and then perform low-pass filtering on the motion vector of the grid point to generate a new grid, and finally transform the image content of each frame to the position of the distorted grid to generate the final output video, so as to achieve video stabilization; another technical solution is to calculate the jitter amplitude value by comparing the static objects in the reference frame and the current frame, and eliminate the jitter of the current frame through the jitter amplitude value; however, these technical solutions can only achieve video stabilization, and cannot solve the problem of the camera's fast and slow motion during shooting.

[0061] In addition, there is another technical solution, which is to obtain the motion data of the feature points by matching the feature points of adjacent video frames, then calculate the smooth motion data based on the motion data of the feature points, and finally correct the video frames based on the smooth motion data. However, this technical solution requires analyzing the image content and estimating the camera motion by matching the feature points. This processing method has a large computational overhead and is not suitable for mobile devices that require real-time calculations. In addition, since the method that relies on image content is easily affected by the imaging quality, different light intensity, light color, and image clarity will affect the final estimation result. Therefore, the accuracy of estimating camera motion based on image content is low.

[0062] Based on one or more problems arising in the related art, the present disclosure first provides a method for uniformly displaying videos. The following uses the execution method of a terminal device as an example to specifically describe the method for uniformly displaying videos in an exemplary embodiment of the present disclosure.

[0063] Figure 4 The flow chart of a method for uniformly displaying a video in this exemplary embodiment includes the following steps S410 to S440:

[0064] In step S410, a current image frame and current camera posture data corresponding to the current image frame are obtained.

[0065] In an exemplary embodiment, the current image frame refers to the image frame corresponding to the current moment when the terminal device is shooting the target video, and the current camera posture data refers to the data used to characterize the rotation state of the camera of the terminal device when shooting the current image frame. For example, the current camera posture data may be the inertial measurement data of the camera when shooting the current image frame. The inertial measurement data may include the angular velocity data of the camera along three axes. The current camera posture data may also be the gravity acceleration data used to characterize the camera when shooting the current image frame. This exemplary embodiment does not make any special limitation on this.

[0066] In step S420, a camera posture data sequence is constructed according to the current camera posture data and the camera posture data corresponding to a plurality of image frames after the current image frame.

[0067] In an exemplary embodiment, the multiple image frames after the current image frame refer to image frames corresponding to multiple moments after the current moment. The multiple image frames after the current image frame may be image frames corresponding to multiple consecutive moments after the current moment, or may be image frames corresponding to multiple discontinuous moments after the current moment. For example, assuming that the current moment may be t, then the multiple consecutive image frames after the current image frame may be m image frames corresponding to moment t+1, moment t+2,..., t+m+1, where m may be a positive integer greater than or equal to 60, or may be a positive integer greater than or equal to 90. The specific value of m may be customized according to the actual application scenario. Of course, it may also be image frames corresponding to moment t+1, moment t+3, moment t+7,..., but this exemplary embodiment is not limited to this.

[0068] In step S430, smoothed camera pose data corresponding to the current image frame is determined through the camera pose data sequence.

[0069] In an exemplary embodiment, the smoothed camera posture data refers to data used to characterize the rotation posture that the camera should be in when the target video corresponding to the current image frame moves at a uniform speed.

[0070] In step S440, the current image frame is processed at a uniform speed based on the current camera posture data and the smoothed camera posture data to obtain a current image frame after uniform speed anti-shake.

[0071] In an exemplary embodiment, the uniform speed processing refers to a process of cropping the display content of the current image frame under the current camera posture so that the display content conforms to the smooth camera posture.

[0072] The current image frame after uniform speed anti-shake refers to an image frame corresponding to a smooth camera posture obtained after uniform speed processing of the current image frame, and a video displayed at a uniform speed is formed by multiple image frames after uniform speed anti-shake.

[0073] Steps S410 to S440 are described in detail below.

[0074] In an exemplary embodiment, inertial measurement data collected when shooting the current image frame with the current camera posture can be obtained. The inertial measurement data refers to data measured by an IMU unit. The IMU unit can measure the angular velocity of the camera moving along three axes at the current moment.

[0075] After obtaining the inertial measurement data corresponding to the current camera posture when shooting the current image frame, the Euler angle of the current camera posture can be determined according to the inertial measurement data. Specifically, the Euler angle between the camera posture at the current moment and the camera posture at the initial moment or the camera posture at the previous moment can be obtained by integrating the angular velocity output by the IMU unit.

[0076] Quaternions refer to a type of hypercomplex number, which is composed of a real number plus three imaginary units i, j and k. [i, j, k] is the imaginary axis unit vector. The geometric meaning of i, j and k themselves can be understood as a rotation, where i rotation represents the rotation from the positive direction of the Z axis to the positive direction of the Y axis in the plane where the Z axis intersects the Y axis, j rotation represents the rotation from the positive direction of the X axis to the positive direction of the Z axis in the plane where the X axis intersects the Z axis, and k rotation represents the rotation from the positive direction of the Y axis to the positive direction of the X axis in the plane where the Y axis intersects the X axis. -i, -j, and -k represent the reverse rotations of i, j, and k rotations, respectively.

[0077] The transformation from one coordinate system to another can be achieved by three consecutive rotations around different coordinate axes. The three rotation angles are collectively called Euler angles. Euler angles consist of three rotations around an axis, and the order of these three rotations is arbitrary. The rotation axis can be of the reference system or the camera system. Euler angles are not directly equal to the camera posture, but only describe three rotations around an axis. Euler angles can be used to describe the conversion relationship between two camera postures.

[0078] In this example embodiment, the rotation order of the Euler angle can be defined as Z axis-Y axis-X axis. Assuming ω, γ represents the rotation angle around the Z axis, Y axis, and X axis respectively. The camera is at t 1 The posture at the moment is P 1 , t 2 The posture at the moment is P 2 , then from P 1 To P 2 The Euler angles can be expressed as Euler Angles It can be expressed as 1 The camera is in the camera posture P 1 First rotate ω around the Z axis, then rotate around the Y axis Finally, rotate γ around the X axis to reach the camera posture P 2 .

[0079] For example, suppose that the Euler angle of the camera posture at the current moment of capturing the current image frame relative to the camera posture at the initial moment is P, and Since the use of Euler angles to describe the camera posture will cause the gimble lock problem, in order to more conveniently describe the camera posture, in this example embodiment, the Euler angles are converted into quaternions to describe the camera posture. Assuming that the quaternion corresponding to the Euler angle P is q, the quaternion can be expressed as equation (1):

[0080]

[0081] Among them, w, x, y, and z can respectively represent four channel values ​​of the quaternion. Specifically, w can represent the real part of the quaternion q, and x, y, and z can represent the imaginary part of the quaternion q.

[0082] The Euler angles can be converted to obtain a quaternion corresponding to the current camera posture, and the converted quaternion can be used as the current camera posture data corresponding to the current camera posture describing the current image frame. According to the same conversion process, the quaternion corresponding to the camera posture of multiple image frames after the current image frame can be determined, and the converted quaternion can be used as the camera posture data of multiple image frames after the current image frame.

[0083] In an exemplary embodiment, the camera pose data may include a quaternion for describing the camera pose, which may be obtained by Figure 5 The steps in the above code are used to determine the smoothed camera pose data corresponding to the current image frame through the camera pose data sequence. Figure 5 As shown, it may specifically include:

[0084] Step S510, determining the current camera motion speed by fitting each quaternion in the camera posture data sequence;

[0085] Step S520, performing sliding average processing on the current camera motion speed to obtain a smoothed camera motion speed;

[0086] Step S530: determining smoothed camera posture data corresponding to the current image frame according to the smoothed camera motion speed.

[0087] The current camera motion speed refers to the camera motion speed of the current image frame. In this embodiment, the quaternions in the camera posture data sequence are fitted. Specifically, each channel of each quaternion (i.e., w, x, y, z in equation (1)) is fitted respectively to obtain a fitting straight line corresponding to each channel. It is easy for a person skilled in the art to understand that the slope of the fitting straight line corresponding to each channel is the camera motion speed of the current image frame.

[0088] Sliding average processing refers to the process of weighted iterative processing of the current camera motion speed at each moment. The sliding average processing can make the changes of the current camera motion speed at each moment smoother. The smoothed camera motion speed refers to the camera motion speed obtained by sliding average processing of the current camera motion speed at the current moment. Compared with the current camera motion speed, the smoothed camera motion speed can make the adjusted video smoother.

[0089] For example, suppose the current camera pose data of the current image frame is the quaternion q i , the camera posture data corresponding to the consecutive image frames after the current image frame is the quaternion q i+1 , quaternion q i+2 ······Quaternion q i+m-1 , then the camera posture data sequence can be expressed as:

[0090]

[0091] Then you can Each channel of each quaternion is fitted, and the fitted straight line can be expressed as equation (2):

[0092]

[0093] Among them, w, x, y, and z can represent the four channel values ​​of the quaternion respectively. can represent the slope of the fitted straight line respectively, It can represent the current camera movement speed. can respectively represent the intercept of the fitting straight line obtained by fitting, which is a constant.

[0094] In this example embodiment, the current camera motion speed is continuously subjected to sliding average processing, that is, The sliding process can be expressed as equation (3):

[0095]

[0096] in, can represent the slope after sliding average, Can represent smooth camera motion speed, can represent the smooth camera motion speed at the previous moment, and σ can represent a preset weight. For example, σ can be 0.95 or 0.9. The specific value of σ can be customized according to the actual application situation, and this example embodiment does not make any special limitation on this.

[0097] Getting smooth camera motion speed Afterwards, the smoothed camera pose data of the current image frame can be predicted by the smoothed camera motion speed. For example, the smoothed camera pose data can be expressed by equation (4):

[0098]

[0099] Among them, q′ can represent the current time t i Smoothed camera pose data, The four channel values ​​of the smoothed camera posture data q′ can be represented respectively, can represent the last time t i-1 The four channel values ​​of the smoothed camera pose data.

[0100] Of course, the above is merely an illustrative example and should not impose any special limitations on this exemplary embodiment.

[0101] In an exemplary embodiment, the current image frame after uniform speed anti-shake can be cropped through the step 6 to ensure the display integrity of the current image frame after uniform speed anti-shake. Figure 6 As shown, it may specifically include:

[0102] Step S610, obtaining a preset cropping ratio, and determining a cropping area under the smooth camera posture represented by the smooth camera posture data according to the cropping ratio;

[0103] Step S620, detecting whether the cropping area is within the current image frame;

[0104] Step S630: if the cropped area is not within the current image frame, it is determined that the smooth camera posture has a limit collision phenomenon, and the smooth camera posture data is updated;

[0105] Step S640: If the cropping area is within the current image frame, it is determined that the smooth camera posture does not have a limit collision phenomenon, and the current image frame after uniform speed anti-shake is cropped and output according to the cropping area.

[0106] Among them, the cropping ratio refers to the pre-set data used to crop the adjusted current image frame. By cropping the adjusted current image frame with the preset cropping ratio, the integrity of the display content of the current image frame in the final output can be ensured. The cropping ratio can be determined according to the size of the current image frame, or a fixed cropping ratio can be set. This example embodiment does not make any special limitations on this.

[0107] The cropping area refers to the area that needs to be cropped and output for the current image frame under the smooth camera posture, which is determined according to the cropping ratio.

[0108] The limit collision phenomenon is defined in this disclosure as that in some shooting scenes with more drastic motion changes, some areas of the new image frame obtained after adjusting the current camera posture are not the imaging range of the original image frame. When the limit collision phenomenon occurs, it is impossible to convert the current image frame to obtain a new image frame corresponding to the adjusted camera posture, so the limit collision phenomenon needs to be avoided.

[0109] If it is detected that the cropped area is not within the current image frame, it can be considered that some areas of the new image frame under the smooth camera posture are not the imaging range of the current image frame. At this time, a suitable new image frame cannot be obtained based on the current image frame. That is, it is determined that the smooth camera posture represented by the smooth camera posture data has a limit collision phenomenon, and the smooth camera posture needs to be re-determined.

[0110] If it is detected that the cropped area is within the current image frame, it can be considered that the content area of ​​the new image frame under the smooth camera posture is within the imaging range of the current image frame, and the new image frame under the smooth camera posture can be obtained based on the current image frame, that is, it is determined that the smooth camera posture represented by the smooth camera posture data does not have a limit collision phenomenon, and the current image frame after uniform speed anti-shake can continue to be cropped and output according to the cropped area to obtain a new image frame with a complete display, that is, the image frame after uniform speed anti-shake processing.

[0111] By setting up the detection of limit collision phenomenon, unreasonable camera posture adjustment can be discovered in time, the accuracy and completeness of the output results can be ensured, the sudden change of the output video can be reduced, the smoothness of the video after uniform speed output can be further improved, and the video quality can be improved.

[0112] In an exemplary embodiment, step S630 may include steps S710 to S740 to update the smoothed camera posture data. Figure 7 As shown, it may specifically include:

[0113] Step S710, analyzing multiple image frames after the current image frame frame by frame until a target image frame with a limit collision phenomenon is detected;

[0114] Step S720, performing a binary search with the original camera pose and the original smooth camera pose of the target image frame as endpoints to determine a critical smooth camera pose without a limit collision phenomenon;

[0115] Step S730, updating the smooth camera motion speed corresponding to the current image frame according to the critical smooth camera posture;

[0116] Step S740, determining the smoothed camera pose data corresponding to the current image frame by using the updated smoothed camera motion speed, to obtain updated smoothed camera pose data.

[0117] Among them, the target image frame refers to an image frame in which a limit collision phenomenon occurs among multiple image frames after the current image frame. The original camera pose refers to the camera pose represented by the original camera pose data corresponding to the target image frame. The original smooth camera pose refers to the camera pose represented by the original smooth camera pose data calculated from the original camera pose data. The original camera pose data corresponding to the target image frame can be calculated by equation (1). The original smooth camera pose data can be calculated by equation (2), equation (3) and equation (4). The specific calculation process refers to the aforementioned embodiment and will not be repeated here.

[0118] The critical smooth camera pose refers to a camera pose where there is just no collision limit phenomenon between the original camera pose and the original smooth camera pose. Specifically, the original camera pose and the original smooth camera pose of the target image frame can be used as two endpoints of the binary search until the critical smooth camera pose is found.

[0119] Since the smooth camera posture corresponding to the current image frame has a limit collision phenomenon, it means that the smooth camera posture data calculated at this time is not suitable for uniform speed processing of the current image frame, so it needs to be updated and adjusted. In order to ensure that the updated smooth camera posture data will clearly not have a limit collision phenomenon, this example embodiment not only determines whether the current image frame has a limit collision phenomenon, but also determines whether the image frames in the future with the current motion trend (i.e., multiple image frames after the current image frame) have a limit collision phenomenon. If the limit collision phenomenon exists in the future image frames, the camera motion speed can be adjusted in advance, that is, the smooth camera posture data is updated to make the camera speed change more uniform.

[0120] The smooth camera motion speed corresponding to the current image frame can be updated according to the critical smooth camera posture. For example, the smooth camera motion speed can be updated by using equations (5) and (6):

[0121]

[0122]

[0123] Among them, Δ w , Δ x , Δ y , Δ z It can represent the intermediate parameters used to update the smooth camera motion speed corresponding to the current image frame, The four channel values ​​of the quaternion corresponding to the critical smooth camera posture can be represented. It can represent the four channel values ​​of the quaternion corresponding to the camera posture of the image frame at the previous moment of the current moment t, t i+c It can represent the time corresponding to the critical smooth camera pose, t i-1 It can represent the previous moment of the current moment t. can represent the updated smooth camera motion speed, It can represent the smooth camera motion speed at the previous moment, and ε can represent the preset weight data. For example, ε can be 0.9 or 0.8. The specific value can be customized according to the actual application situation, and this example embodiment does not make any special limitation on this.

[0124] The updated smooth camera motion speed can be calculated by using equations (5) and (6), and then the smooth camera pose data corresponding to the current image frame can be re-determined according to the updated smooth camera motion speed and equation (4) to obtain the updated smooth camera pose data.

[0125] By detecting whether there is a limit collision phenomenon in future image frames and determining the critical smooth camera posture of future image frames, returning to update and adjust the smooth camera posture data of the current image frame, it can further ensure that the current image frame and future image frames will not have a limit collision phenomenon, effectively improve the accuracy of the video after uniform speed, make the camera movement speed smoother, and improve the display quality of the uniform speed output video.

[0126] Further, after determining the updated smoothed camera posture data, you can continue to Figure 8 The steps shown are used to detect the updated smoothed camera pose data, refer to Figure 8 As shown, it may specifically include:

[0127] Step S810, detecting whether the updated smoothed camera posture data has a limit collision phenomenon;

[0128] Step S820, if the updated smooth camera posture data has a limit collision phenomenon, a binary search is performed with the current camera posture and the updated smooth camera posture as endpoints to determine a target smooth camera posture that does not have a limit collision phenomenon;

[0129] Step S830: Using the camera pose data corresponding to the target smooth camera pose as updated smooth camera pose data.

[0130] The target smooth camera pose refers to a camera pose that is found by binary search between the current camera pose of the current image frame and the smooth camera pose represented by the updated smooth camera pose data and that just does not have a limit collision phenomenon.

[0131] Below is the image frame I i+c Taking the example of FIG. 1 as an example, the detection and adjustment strategy of the limit collision phenomenon in this exemplary embodiment is described as follows:

[0132] First, for the consecutive multiple image frames {q i+c ,q i+c+1 ,…,q i+m-1}, the slope k is obtained by linear fitting through equation (2): i+c ;

[0133] Second, according to the slope k i+c And the relationship (3) calculates the smooth camera motion speed l i+c ;

[0134] Third, according to the smooth camera motion speed l i+c And the relationship (4) predicts the smoothed camera pose data q′ i+c ;

[0135] Fourth, judge q′ i+c Whether the corresponding camera posture has a limit collision phenomenon: if q′ i+c If the corresponding camera posture does not have a limit collision phenomenon, then return to the first step and continue to judge the next image frame I i+c+1 ; if q′ i+c If the corresponding camera posture has a limit collision phenomenon, continue to execute the fifth step;

[0136] Fifth, with q i+c The corresponding camera pose and q′ i+c The corresponding camera postures are used as two endpoints, and the binary search method is used to find the critical smooth camera posture H that just does not have the limit collision phenomenon. i+c , binary search can be used because q i+c There is no limit collision phenomenon, q′ i+c If there is a limit collision phenomenon, then H i+c In q i+cand q′ i+c between; and, through H i+c Equations (5) and (6) update the smooth camera motion speed l i ;

[0137] Sixth, use the updated smooth camera motion speed l i And equation (4) predicts the smooth camera motion pose data of the current image frame, that is, the updated smooth camera motion pose q′ i ; Continue to determine the updated smooth camera motion posture q′ i Is there a limit collision phenomenon? If the updated smooth camera motion posture q′ i There is a limit collision phenomenon, continue with the current camera movement posture q i and q′ i Use binary search for the two endpoints to find the target smooth camera pose H that just does not hit the limit i , and the camera posture data corresponding to the target smooth camera posture is used as the updated smooth camera posture data, that is, let q′ i =H i , and exit the current limit collision detection and adjustment process. The following image frames {q i+c+1 ,…,q i+m-1 No need to judge any further.

[0138] In an exemplary embodiment, the current camera pose data may include a current quaternion, and the smoothed camera pose data may include a smoothed quaternion, which may be obtained by Fig. 9 The steps in the implementation of the uniform speed processing of the current image frame based on the current camera posture data and the smoothed camera posture data, refer to Fig. 9 As shown, it may specifically include:

[0139] Step S910, determining a rotation quaternion according to the conjugate quaternion of the current quaternion and the smoothed quaternion;

[0140] Step S920, based on a preset projection method, transform and project the current video image in the two-dimensional coordinate system into a three-dimensional coordinate system to obtain a projection coordinate set;

[0141] Step S930, rotating the points in the projection coordinate set according to the rotation quaternion to obtain a uniform projection coordinate set;

[0142] Step S940, back-projecting the uniformly-speeded projection coordinate set according to the projection method to obtain a current image frame after uniformly-speed anti-shake.

[0143] The rotation quaternion refers to a rotation relationship determined based on the conjugate quaternion of the current quaternion and the smooth quaternion. For example, assuming that the current quaternion can be expressed as q i , the conjugate quaternion of the current quaternion can be expressed as The smoothed quaternion can be expressed as q′ i , then the rotation quaternion can be

[0144] The preset projection method refers to a processing method used to transform and project the pixel coordinates of the current video image in a two-dimensional coordinate system into a three-dimensional coordinate system. For example, the preset projection method can be a camera pinhole imaging model or a coordinate transformation matrix. This example embodiment does not specifically limit this.

[0145] For example, a 2D plane image can be projected onto a 3D spherical surface through a preset projection method, such as a camera pinhole imaging model. Assuming that the pixel coordinates on the 2D plane image taken by the camera are (x, y), the corresponding pixel coordinates projected onto the 3D spherical surface through the camera pinhole imaging model are (X, Y, Z), i.e., the projection coordinate set. Then, the pixel coordinates (X, Y, Z) on the 3D spherical surface are rotated using the quaternion Rotate to the new pixel coordinates (X′, Y′, Z′), and finally project the pixel coordinates (X′, Y′, Z′) on the 3D spherical surface back to the 2D plane through the camera pinhole imaging model to obtain (x′, y′). i Each pixel on the image can be rotated using the above axis rotation method to obtain a new image frame I′ i , where I′ i is q′ i The video frame corresponding to the posture, that is, I′ i is the current image frame after uniform speed anti-shake, and then I′ is cropped according to the cropping ratio i Output after cutting.

[0146] Fig.10 A schematic diagram schematically illustrates a principle of implementing uniform speed anti-shake in an exemplary embodiment of the present disclosure.

[0147] refer to Fig.10 As shown, step S1010, through a preset projection method, such as through a camera pinhole imaging model, the pixel point coordinates (x, y) on the current image frame 1001 are projected onto the 3D spherical surface 1002 to obtain the pixel point coordinates (X, Y, Z);

[0148] Step S1020, spatially rotating the pixel point coordinates (X, Y, Z) on the 3D spherical surface 1002 by using the calculated rotation quaternion to obtain new pixel point coordinates (X′, Y′, Z′) on the 3D spherical surface 1003;

[0149] In step S1030, the new pixel coordinates (X′, Y′, Z′) on the 3D spherical surface 1003 are back-projected according to a preset projection method, such as through a camera pinhole imaging model, to obtain the pixel coordinates (x′, y′), and the current image frame after uniform speed stabilization is obtained through the new pixel coordinates (x′, y′), and the current image frame after uniform speed stabilization is cropped to obtain the final output image frame 1004.

[0150] Fig.11 A schematic diagram of a process of displaying a video at a uniform speed in an exemplary embodiment of the present disclosure is schematically shown.

[0151] refer to Fig.11 As shown, in step S1110, the collected inertial measurement data is integrated to obtain the Euler angles from the image frame at the current moment to multiple image frames at the future moment;

[0152] Step S1120, converting the Euler angle into a quaternion;

[0153] Step S1130, performing linear fitting on each channel of the quaternion to obtain the current camera motion speed;

[0154] Step S1140, performing sliding average processing on the current camera motion speed to obtain a smoothed camera motion speed;

[0155] Step S1150, predicting the smooth camera posture of the camera at the current moment according to the smooth camera motion speed;

[0156] Step S1160, performing a limit collision phenomenon detection on the smooth camera posture, and if a limit collision phenomenon exists, updating the smooth camera posture through a limit collision phenomenon detection adjustment strategy;

[0157] Step S1170, based on the current camera posture data and the updated smoothed camera posture data, the current image frame is processed at a uniform speed to obtain a current image frame after uniform speed anti-shake;

[0158] Step S1180, cropping and outputting the current image frame after uniform speed anti-shake, and ending the current process.

[0159] In summary, in this exemplary embodiment, the current image frame and the current camera posture data corresponding to the current image frame are obtained, and the camera posture data sequence can be formed according to the current camera posture data and the camera posture data corresponding to multiple image frames after the current image frame, and then the smoothed camera posture data corresponding to the current image frame can be determined through the camera posture data sequence, and finally the current image frame is processed at a uniform speed based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake. On the one hand, the smoothed camera posture data of the current image frame is calculated according to the camera posture data of multiple frames, and the uniform speed processing of the video is realized by the smoothed camera posture data. The processing result is not affected by the imaging quality of the camera, the error is effectively reduced, and the processing result is more stable; on the other hand, the smoothed camera posture data corresponding to the current image frame is determined according to the current camera posture data of the current image frame and the camera posture data corresponding to multiple image frames after the current image frame, the accuracy of the smoothed camera posture data is improved, and the smoothness and consistency of the uniform speed result in the entire processing process are guaranteed, the video quality after uniform speed processing is improved, and the user experience is improved.

[0160] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0161] For further reference, Fig.12 As shown, in the implementation of this example, a video uniform speed display device 1200 is also provided, which may include a current camera posture acquisition module 1210 , a camera posture sequence construction module 1220 , a smooth camera posture determination module 1230 and an image frame uniform speed module 1240 .

[0162] in:

[0163] The current camera posture acquisition module 1210 is used to acquire the current image frame and the current camera posture data corresponding to the current image frame;

[0164] The camera posture sequence construction module 1220 is used to construct a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to multiple image frames after the current image frame;

[0165] The smooth camera posture determination module 1230 is used to determine the smooth camera posture data corresponding to the current image frame through the camera posture data sequence;

[0166] The image frame uniform speed module 1240 is used to perform uniform speed processing on the current image frame based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake.

[0167] In an exemplary embodiment, the camera pose data may include a quaternion for describing the camera pose, and the smoothed camera pose determination module 1230 may be used to:

[0168] Determine the current camera motion speed by fitting each quaternion in the camera posture data sequence;

[0169] Performing sliding average processing on the current camera motion speed to obtain a smoothed camera motion speed;

[0170] Determine smoothed camera pose data corresponding to the current image frame according to the smoothed camera motion speed.

[0171] In an exemplary embodiment, the current camera posture acquisition module 1210 may be used to:

[0172] Acquire inertial measurement data collected when the current image frame is captured with a current camera posture, and determine the Euler angles of the current camera posture according to the inertial measurement data;

[0173] Converting the Euler angles to obtain a quaternion corresponding to the current camera posture, and using the quaternion as current camera posture data corresponding to the current image frame; and

[0174] Determine camera posture data corresponding to multiple image frames after the current image frame.

[0175] In an exemplary embodiment, the video uniform speed display device 1200 may include a cropping unit, which may be used to:

[0176] Obtaining a preset cropping ratio, and determining a cropping area under the smooth camera posture represented by the smooth camera posture data according to the cropping ratio;

[0177] Detecting whether the cropping area is within the current image frame;

[0178] If the cropped area is not within the current image frame, determining that the smooth camera posture has a limit collision phenomenon, and updating the smooth camera posture data;

[0179] If the cropping area is within the current image frame, it is determined that there is no limit collision phenomenon in the smooth camera posture, and the current image frame after uniform speed anti-shake is cropped and output according to the cropping area.

[0180] In an exemplary embodiment, the cutting unit may also be used to:

[0181] Analyze multiple image frames after the current image frame frame by frame until a target image frame with a limit collision phenomenon is detected;

[0182] Perform a binary search using the original camera pose and the original smooth camera pose of the target image frame as endpoints to determine a critical smooth camera pose without a limit collision phenomenon;

[0183] updating the smooth camera motion speed corresponding to the current image frame according to the critical smooth camera posture;

[0184] The smoothed camera posture data corresponding to the current image frame is determined according to the updated smoothed camera motion speed to obtain the updated smoothed camera posture data.

[0185] In an exemplary embodiment, the cutting unit may also be used to:

[0186] Check whether the updated smooth camera posture data hits the limit;

[0187] If the updated smooth camera posture data has a limit collision phenomenon, a binary search is performed with the current camera posture and the updated smooth camera posture as endpoints to determine a target smooth camera posture that does not have a limit collision phenomenon;

[0188] The camera posture data corresponding to the target smooth camera posture is used as the updated smooth camera posture data.

[0189] In an exemplary embodiment, the current camera pose data may include a current quaternion, and the smoothed camera pose data may include a smoothed quaternion; the image frame constant speed module 1240 may also be used to:

[0190] Determine a rotation quaternion according to the conjugate quaternion of the current quaternion and the smoothed quaternion;

[0191] Based on a preset projection method, the current video image in the two-dimensional coordinate system is transformed and projected into a three-dimensional coordinate system to obtain a projection coordinate set;

[0192] Rotate the points in the projection coordinate set according to the rotation quaternion to obtain a uniform projection coordinate set;

[0193] The uniformly-speeded projection coordinate set is back-projected according to the projection method to obtain a current image frame after uniformly-speeded anti-shake.

[0194] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0195] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0196] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0197] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0198] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0199] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0200] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0201] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for uniformly displaying a video, characterized in that: include: Acquire a current image frame and current camera posture data corresponding to the current image frame; wherein the camera posture data includes a quaternion for describing the camera posture; Constructing a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to a plurality of image frames after the current image frame; Determine the current camera motion speed by fitting each quaternion in the camera posture data sequence; perform sliding average processing on the current camera motion speed to obtain a smoothed camera motion speed; determine the smoothed camera posture data corresponding to the current image frame according to the smoothed camera motion speed; The current image frame is processed at a uniform speed based on the current camera posture data and the smoothed camera posture data to obtain a current image frame after uniform speed anti-shake.

2. The method according to claim 1, characterized in that The obtaining of current camera posture data corresponding to the current image frame includes: Acquire inertial measurement data collected when the current image frame is captured with a current camera posture, and determine the Euler angles of the current camera posture according to the inertial measurement data; Converting the Euler angles to obtain a quaternion corresponding to the current camera posture, and using the quaternion as current camera posture data corresponding to the current image frame; and Determine camera posture data corresponding to multiple image frames after the current image frame.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining a preset cropping ratio, and determining a cropping area under the smooth camera posture represented by the smooth camera posture data according to the cropping ratio; Detecting whether the cropping area is within the current image frame; If the cropped area is not within the current image frame, determining that the smooth camera posture has a limit collision phenomenon, and updating the smooth camera posture data; If the cropping area is within the current image frame, it is determined that there is no limit collision phenomenon in the smooth camera posture, and the current image frame after uniform speed anti-shake is cropped and output according to the cropping area.

4. The method according to claim 3, characterized in that The updating of the smoothed camera posture data comprises: Analyze multiple image frames after the current image frame frame by frame until a target image frame with a limit collision phenomenon is detected; Perform a binary search using the original camera pose and the original smooth camera pose of the target image frame as endpoints to determine a critical smooth camera pose without a limit collision phenomenon; updating the smooth camera motion speed corresponding to the current image frame according to the critical smooth camera posture; The smoothed camera posture data corresponding to the current image frame is determined according to the updated smoothed camera motion speed to obtain the updated smoothed camera posture data.

5. The method according to claim 4, characterized in that The updating of the smoothed camera posture data comprises: Check whether the updated smooth camera posture data hits the limit; If the updated smooth camera posture data has a limit collision phenomenon, a binary search is performed with the current camera posture and the updated smooth camera posture as endpoints to determine a target smooth camera posture that does not have a limit collision phenomenon; The camera posture data corresponding to the target smooth camera posture is used as the updated smooth camera posture data.

6. The method according to claim 1, characterized in that The current camera pose data includes a current quaternion, and the smoothed camera pose data includes a smoothed quaternion; The uniform speed processing of the current image frame based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake includes: Determine a rotation quaternion according to the conjugate quaternion of the current quaternion and the smoothed quaternion; Based on a preset projection method, the current image frame in the two-dimensional coordinate system is transformed and projected into a three-dimensional coordinate system to obtain a projection coordinate set; Rotate the points in the projection coordinate set according to the rotation quaternion to obtain a uniform projection coordinate set; The uniformly-speeded projection coordinate set is back-projected according to the projection method to obtain a current image frame after uniformly-speeded anti-shake.

7. A video uniform speed display device, characterized in that: include: A current camera posture acquisition module, used to acquire a current image frame and current camera posture data corresponding to the current image frame; wherein the camera posture data includes a quaternion for describing the camera posture; A camera posture sequence construction module, used to construct a camera posture data sequence according to the current camera posture data and the camera posture data corresponding to a plurality of image frames after the current image frame; A smooth camera attitude determination module is used to determine the current camera motion speed by fitting each quaternion in the camera attitude data sequence; perform sliding average processing on the current camera motion speed to obtain a smooth camera motion speed; and determine the smooth camera attitude data corresponding to the current image frame according to the smooth camera motion speed; The image frame uniform speed module is used to perform uniform speed processing on the current image frame based on the current camera posture data and the smoothed camera posture data to obtain the current image frame after uniform speed anti-shake.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.

Images